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(54) Title: SYSTEM FOR OBJECT VERIFICATION AND IDENTIFICATION 
(57) Abstract 



The present system identifies an object, 
such as a face, from unique signals and values 
emitted by the attributes of components on 
the object such as eyes, eyebrows, hairline, 
mouth, cheeks, ears, and chin (22). The object 
image is analyzed to determine the different 
components (325). These components are 
then transformed (340) through one of several 
transformations such as geometric, cosine, or 
Kohonen*s, to yield a unique* description for 
each component. The data obtained is then 
transferred to either a portable memory, ID 
card or data base. A decision process of 
comparing the two results, from the object 
image, and from the data encoded in the ID 
card or data base is subsequently performed. 
In one example, this technology identifies 
human faces to verify the authenticity of an 
identity and then to grant or deny authority to 
carry out actions. 
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SYSTEM FOR OBJECT VERIFICATION AND IDENTIFICATION 

This application is a continuation-in-part of 
application Serial No. 08/507,528, filed September 26, 
1995, pending, the entire disclosure of which is 
^ incorporated herein by reference. 

FIELD OF INVENTION 
This invention relates generally to information 
storage and retrieval computer and camera systems for 
obtaining and storing certain data relating to specific 
10 images and providing rapid access to that data for 
purposes of object targeting, enrollment, 
identification, verification, and classification. 

BACKGROUND OF THE INVENTION 

15 Many methods have been developed for 

commercial, military, and medical visual inspection 
applications. One particular area of interest for 
recent development efforts has been the area of image 
recognition. The basic idea behind image recognition 

20 is to provide a system that captures data relating to* 
an image area of an object or person and then compares 
that captured image data to information stored in a 
computer memory. 

However, many image recognition systems have 

25 drawbacks. First, they are characteristically slow, 
and subject to serious recognition and/or 
identification errors. This is, in part, due to the 
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background of the art. The fact that prior methods 
have, for example, processed the entire human face, and 
have^ used quantitative data obtained to effect the 
process of identification has resulted in very complex 
i processor-bound systems. As a consequence, these 
systems are prone to error, and are cumbersome and 
costly. For example, the predominant facial 
recognition systems, which are Image Recognition, 
Pattern Recognition, and Retinal Recognition all rely 
on a central data storage, in the form of databases, 
which contain either a computerized copy of the entire 
facial image of the persons who are to be identified, 
or some abstracted form of the entire face or portions 
of the face to be reconnected later. The data for such 
15 systems is substantial for each face. Retrieval of 

data for these systems is time consuming, particularly 
as the number of enrolled persons increases. 

An example of a system exhibiting these 
drawbacks is Kodak's card-based facial security system. 
20 As understood, the Kodak system classifies fifty areas 
of the card holder's face, identifying each area with a 
2-byte code. That data is then stored on the stripe of 
a magnetic card. The card user must then have his/her 
face compared with the stored facial code in order for 
25 a match to be made. A drawback to this technique is 
that it requires the local computer to recognize 
multiple areas of the face, then classify each of these 
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areas, and then compare the instant classification to 
the code stored on the card. Hence, a significant 
processing burden is required at the recognition 
station. Moreover, the Kodak system is relatively 
5 inflexible in the sense that it is limited only to 

those things that have been classified. Thus, for the 
Kodak system to operate on other objects, a whole new 
classification scheme needs to the developed. The 
classification effort is a significant, labor-intensive 
10 process. 

Other known verification processes involve the 
comparison of large image structures stored in the 
system's database. The techniques used are various, 
but essentially work with the same data sets. The 

15 costs involved are high due to the complexity of the 
databases, the large size requirements for physical 
storage devices to contain these databases, and the 
necessity for fast computing speeds to process the 
information for real-time use. 

20 Object recognition against a known data set is 

a relatively easy art for humans to master, but 
presents difficult problems for computers. For a 
technique that permits a computer to carry out a 
successful recognition process, the process must yield 

25 quantifiable numerical descriptors. This process of 
checking against the known data set would, of course, 
also involve a quantifiable registration and enrollment 
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of the object. The data set generated during the 
enrollment process is used for checking against the 
generated data from the recognition and verification 
systems. Yet, the numerical data produced by the 
5 digitized object images, such as human facial images, 
can be large. Once about a thousand complete images 
are stored, the retrieval and comparison of the data 
sets can become uneconomical and time-consuming. 

To avoid these drawbacks, certain systems, such 
10 as exemplified by U.S. Patent No. 5,161,204 or U.S. 
Patent No. 5,164,992, have resorted to complex 
classification schemes in order to extract key features 
and to perform discrete calculations for identification 
and verification. However, such schemes increase the 
15 processor overhead that is necessary to extract 
components from the image field. They do not 
necessarily reduce the storage requirements necessary 
to retain a database of captured faces. Thus, there is 
a need in the art for a verification and identification 
system that allows for high reliability, high accuracy 
and low cost, but at the same time is efficient. 



20 



SUMMARY OF THE INVENTION 

In view of the foregoing, there exists a need 

in the art for a system that does not attempt to 
25 develop identification and/or verification using entire 
object feature comparisons, but instead utilizes 
selected data sets from components of the object for 
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comparing the stored data sets with input images. Such 
combinations of data sets are to be used to uniquely 
identify an object. The independence of the attribute 
data sets leads to a normative analysis that can 
5 compare the data sets generated from input with data 
sets from known objects or persons. 

It is another object of this invention to 
provide a comparison process that involves polling 
various attribute data against a known data set. 

10 It is yet a further object of the invention to 

provide a system and method for obtaining, digitizing, 
and recording selected data sets of separate and 
independent component element image signals, sensed by 
photographic or electronic means, from a person or 

15 object, for subsequent comparison with similarly 

obtained component elements image signals of a person 
or object whereby the degree of similarity between the 
original and the subsequent data sets may be 
determined. 

20 It is yet a further object of the invention to 

provide a method whereby data sets of sensed images may 
be recorded on portable media, such as magnetic stripe 
cards, magnetic discs, printed bar codes, semi-conductor 
devices such as smart cards, or in data bases. 

25 It is an additional object of the invention to 

provide a method" of selecting separate and independent 
component elements of persons or objects whereby the 
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selected components provide image signals for recording 
quantitative structures or arrays of numbers that permit 
statistical identification of the individual person or 
object from among data sets previously obtained from the 
5 same person or similar objects. 

It is yet a further object of the invention to 
provide a method whereby the image signal data sets of 
persons or objects are measured so as to permit the 
determination of a quantitative degree to which the 
10 objects or persons are similar to those originally 
measured. 

It is an additional object of the invention to 
provide a method whereby it may be determined whether 
component image signals obtained from a person or object, 

15 at any time subsequent to the recording of a set of such 
image signals from the person or object, differ, or do 
not differ, from the recorded component images by any, or 
by more than or less than, preselected quantitative 
values, permitting the system to indicate that the 

20 signals may be quantitatively, or qualitatively, 

verified, or not verified, as being from the recorded 
signals of the person or object. 

It is a further object of this invention to 
provide a camera and infra-red LED' array which is 

25 designed to optimize the lighting and imaging of a 
particular area of the object. " " 
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It is another object of the invention to provide 
a system whereby quantitative measures may be selected 
for inclusion in the digital data system for allowing the 
determination of chosen levels of difference between 
5 subsequently obtained component image signals and 

verification of such signals as being, or not being, from 
the recorded signals of a specific person or object. 
- It is another object to provide an 
identification and/ or image verification system that 
10 uses a method of transforming various independent 

attributes of an object, such as, for example, a nose 
on a human face, into a data set for a component. For 
the whole object the various components are independent 
of each other. The data sets are generated during 
15 registration or enrollment. These data sets are 

recorded on a card capable of carrying data, such as a 
magnetic strip or a 2d-barcode, that is issued to a 
holder. In effect, the advantage of the system is that 
instead of maintaining a central database, the 
20 identification data is now decentralized and held on 

the small cards. Also, in the process of transforming 
the object component attributes into data sets, a 
dramatic reduction in the data that is required to be 
identified is achieved. The reduction of the data, 
25 required to uniquely identify complex objects such as 
human faces, also achieves a faster response in the 
identification process. 
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It is a further object of the invention to 
obtain the data sets through various transformations 
that are specific to the object component attribute 
chosen. The comparison process involves the 
5 verification of data present on the ID card, with the 
data sets generated from the video or other image of 
the object that has been registered through an input 
device such as an electronic camera. The comparison 
process utilizes a neural network that has been trained 

10 so as to recognize or identify a particular data set 

(such as human facial image component attributes) . The 
training of the neural network is based on a process of 
polling the various attributes that are obtained at the 
identification station by the computer, against the 

15 component attribute data sets that are present on the 
ID card. The polling assumes that certain distinctive 
features, if in agreement with the data sets on the ID 
card, can override other less distinctive attributes. 
However, as the security needs of the application 

20 increases, the polling process increases the required 
precision of the comparisons. In one embodiment, a 
program is provided for reducing the characteristics of 
an object image, for example, a human face, to a set of 
characteristic numbers; later recalling that set of 

25 characteristics for comparison with an input from an 
"external source. Keys to "this set of numbers are 
encoded as indices and are stored on local source 
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objects, such as 3-3/8" x 2" computer input cards. The 
indices, having been electronically posted to a central 
computer program, point to a second set of data 
retrieved from the computer program. A comparison of 
5 that set then occurs with a second set of similar 

stored data retrieved from a local source, such as the 
card. 

These and other advantages and features of the 
invention, the various embodiments, and other aspects 
10 of the invention, should become more apparent from the 
following description, drawings and claims. 

BRIEF DESCRIPTION OF THE DRAWINGS 

FIG. 1 is a block diagram of the enrollment 
station forming the present invention; 
15 FIG. 2 is a block diagram of the verification 

station forming the present invention; 

FIGS. 3A and 3B are flow-chart diagrams showing 
the functions of the enrollment process; 

FIG. 4 is a flow chart diagram illustrating the 
20 one-on-one preprocessing steps of the enrollment 
process shown in FIGS. 3A-3B; 

FIG. 5 is a flow chart diagram of the 
binarization routine of the enrollment process shown in 
Figs. 3A-3B; 

25 FIG. 6 is a flow chart diagram of a first 

embodiment of the targeting process for the enrollment 
process shown in Figs. 3A-3B; 
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FIG. 7 is a flow chart diagram of the UD and CP 
coordinate estimation functions of the enrollment 
process shown in FIGS. 3A-3B; 

FIG 8 is a flow chart diagram illustrating the 
5 area of interest defining function of the enrollment 
process shown in FIGS . 3A-3B; 

FIG. 9 is a flow chart diagram illustrating the 
normalization procedure of the enrollment process shown 
in Figs. 3A-3B; 

10 FIG. 10 is a flow chart diagram showing the 

transform step of the enrollment process shown in Figs. 
3A-3B; 

FIG. 11 is a flow chart of the second 
transformation process for the enrollment process of 
15 Figs. 3A-3B; 

FIG. 12 is a flow chart of the output coding 
function for the enrollment process of Figs. 3A-3B; 

FIG. 13 is a flow chart illustrating the 
encrypt function for determining useful parameter 
20 vectors for the enrollment process of Figs. 3A-3B; 

FIGS. 14A-14B are flow charts of the process 
for image verification of the present invention; 

FIG. 15 is a flow chart of the image 
verification pre-processing function; 
25 FIG. 16 is a flow chart of the image 

verification setup control function; 
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FIG. 17 is a flow chart of the image 
verification data decryption function; 

FIG, 18 is a flow chart of the image 
verification parameter value comparison function; and 
5 FIG. 19 is a flow chart of the image 

verification identity decision function; 

FIG. 20 is a diagram showing the dimensional 
breakdown of the face; 

FIG. 21 is a top view of the array of infra-red 
10 light emitting diodes used to light a mini-camera 
apparatus; 

FIG. 22 is a perspective transparent diagram of 
the mini-camera and infra-red lighting device and 
components thereof of the invention; 
15 FIG. 23 is a circuit schematic diagram for the 

light array of FIG. 21; 

FIG. 24 is a circuit schematic diagram for the 
mini-camera of FIG. 21; 

FIGS. 25 (a) -(c) are respectively front, side 
20 and perspective views of a first embodiment of a 

housing for the mini-camera arrangement shown in FIG. 
22; 

FIGS. 26 (a) -(c) are respectively front, side 
and perspective views of a second embodiment of a 
25 housing for the mini-camera arrangement shown in FIG. 
22; 
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FIGS. 27 (a) -(c) are respectively front, side 
and perspective views of a third embodiment of the 
housing for the minicamera arrangement of FIG. 22; and 

FIG. 28 is a flow chart illustrating a second 
5 embodiment of the targetting process for the present 
invention. 

DETAILED DESCRIPTION OF THE INVENTION 

Referring now to the drawings wherein like 
reference numbers refer to like parts, the first 

10 embodiment of present system is composed of two 

processes and two hardware systems. The first process 
of this first embodiment is the enrollment process and 
is shown in Figs. 3A-13. The purpose of the enrollment 
process is to code the image of the person or object 

15 and to reduce that image to a portable form, and 

format, such as a card with a magnetic strip, or in a 
database. The second process of the first embodiment 
is shown in Figs. 14A-15. This process is the 
verification process. The verification process 

20 performs the tasks of taking a picture or image of the 
person or object, and comparing the captured image to 
the coded image obtained from the enrollment process. 
If the image of the person or object obtained during 
verification and the coded information obtained from 

?5 the enrollment process match, identity is verified by 
the verification process. The enrollment process and 
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the verification process have elements in common, which 
will be described further below. 

The two hardware systems used in the present 
invention are the enrollment station, shown in Fig. 1, 
5 and the verification station, shown in Fig. 2. 

Fig. 1 is a block diagram of the enrollment 
station 100. The object 10 represents the object that 
will be coded in the enrollment process for later 
verification during the verification process. In a 

10 preferred embodiment, the object 10 is a face of a 
person. However, any object with distinguishing 
features can be used with this invention. For example, 
the object under consideration may constitute a machine 
part under inspection, or a house, or a car key, or an 

15 automobile, or a hand. As another example, in 

warehouse operations, it is essential that, objects are 
correctly labeled, stored, and shipped. The label on 
the object or its container carries data related to 
features of the object. These data permit an automatic 

20 verification that the object is correctly labeled, and 
therefore will be properly stored, packaged, and 
shipped. Object types can vary, as long as an 
identifiable characteristic can be extracted, and is 
stored in the enrollment process. In order to enroll 

25 the person or object, one or more video cameras 2 0 or 
other cameras as set forth in this application and" 
equipped with lighting devices 30, are used to record 
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an image of the object 10. The video camera or cameras 
20 and lighting device 30 are ordinary devices that are 
readily available. In the first embodiment, Panasonic 
camera, CCD Model No. GP-MF602, or similar devices are 
used with either flash or continuous light sources. 
The lighting devices can in this first embodiment 
comprise a ring lamp such as the MICRO FLASH 5000, 
manufactured by VIVITAR Corp., located at Chatworth, 
California, or a standard photography lighting fixture. 
Other camera devices and lighting devices, however, can 
be substituted. For example, a flash device can be 
employed with a Panasonic camera. An example of a 
flash unit is the SUNPAK Softlite 1400M, manufactured 
by TOCAD Company, Tokyo, Japan. Alternatively, a 
continuous incandescent light source can be employed. 
This type of lighting device is particularly useful in 
conjunction with object identification/verification for 
quality control applications. Finally, as will be 
described in Figs. 21-27 of this application, an LED 
lighting device can be employed in conjunction with a 
mini infra-red camera. 

The output of the video camera 20 is connected 
via a port to computer 40. The computer 40 can be a 
personal computer that includes a digitizer card 42 in 
an expansion port. The computer also includes other 
standard components such as a CPU 44, a random access 
memory 4 6, and a permanent storage, in the form of a 
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magnetic disk storage 48, a monitor 50, and a keyboard 
52. 

In the preferred embodiment, a personal 
computer is used having an Intel® Pentium® 
5 microprocessor designed to have a minimum processor 
clock speed of 90 MHz . The computer has in this 
example 32 MBytes of Random Access memory and at least 
1 Gigabyte of static storage. However, any combination 
of clock speed and memory size types can be used in 

10 this system. A conventional hard-drive can be used, 
although other static storage units (e.g., writable 
optical drives, EPROM) are also acceptable. A 
Microsoft Windows® operating system is used. It should 
be noted that the present invention is designed so that 

15 any computer having adequate processor clock speed 

(i.e. preferably a clock speed of at least 60 MHz), and 
a sufficient RAM memory size of (i.e., 16 megabytes of 
PAM) can be employed. 

The digitizer 42 used is a frame grabber card 

20 for transforming camera signals from analog to digital 
form. This card is inserted in the computer 40. In 
the preferred embodiment, the card used is a CX100 
Imagination Board, manufactured by Image Nation 
Corporation, located in Beaverton, Oregon. However, 

25 any digitizer device for video input including direct 
input from digital video cameras can be used with the - 
invention. 
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The output device 60 receives the data from the 
computer 4 0 which is a coded representation of the 
object 10. Device 60 transforms the coded data into an 
appropriate signal and code for placement on a central 
5 storage 72 or portable memory device. The output 

device 60 receives the data from the computer 4 0 which 
is a coded representation of the object 10. Device 60 
transforms the coded data into an appropriate signal 
and code for placement on a central storage 72 or 

10 portable memory device 70. The output device 60 in the 
preferred embodiment is a card reader/writer, or a 
printer. Examples of output devices include magnetic 
or optical reader/writers or smart card reader/writers 
or barcode printers, etc. 

15 In the preferred embodiment, that memory device 

70 is a magnetic stripe card. The output device 60 and 
card 70 are well known in the art. However, other 
portable memory devices such as optical cards, S-RAM 
storage devices, or carriers containing EPROM or UVPROM 

20 memories, can also be used. In addition, known 

barcoding schemes can be employed to bar-code the 
information, where appropriate. 

Fig. 2 is a block diagram of the verification 
station hardware 200. The data of an enrolled person 

25 or object 210 appropriately lighted by lighting 30 will 
be compared to the coded representation of the object 
on the portable memory 70. The object 210 represents 



WO 97/05566 1 7 PCT/US96/1 1996 

the same object as the enrolled object 10 in Fig* 1. 
The purpose of the verification station 200 is to 
output to an appropriate security system, a signal via 
external control 230. For example, the external 
5 control 230 can be an electronic access device, such as 
an electronic door lock, or a motor-driven gate, or any 
other mechanism. The components in the verification 
station 200 are the same as the components of the 
enrollment station 100 with the exception of the input 

10 device 220 and the external control unit 230. 

The card 70 is inserted in the input device 
220, such as a magnetic or optical card or a barcode 
reader, while the object 210 has its image recorded and 
processed by the computer 40. Image recordation and 

15 processing is done in an identical way as in the 

enrollment station discussed above A program in the 
computer 40 for the verification station compares the 
image data of the object 210 with the coded image data 
on the card 70. If there is a satisfactory match, the 

20 verification signal 230 is outputted indicating a 
match, or a failure to match. 

Figs. 3A and 3B are overview flow chart 
diagrams showing the process used by the enrollment 
station 100 in order to encode enrollment data onto the 

25 portable storage medium 70 or central database 72. 

Each of the steps of Figs. 3A-3B are detailed in the 
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flow charts of Figs, 4-13. Each overview step will be 
elaborated on* below by reference to the later figures. 

Cameras 20 provide input for the computer 4 0 
which executes the preprocessing function 302. This 
5 preprocessing function 302 is described in detail in 
Fig. 4. 

Specifically, in Fig. 4, the preprocessing 
function 302 is performed by the computer 40 in 
combination with a frame grabber digitizer 42. The 

10 frame grabber digitizer 42 first transforms the analog 
signal at step 3002, then filters the digitized data at 
step 3004, and enhances the image at step 3006. The 
filtering step 3004 filters out image noise using 
conventionally known techniques. The enhancement step 

15 300 6 enhances image contrast according to the lighting 
conditions, using known techniques. The output of 
these functions is the complete image 305 which is 
composed of a standardized noise-free, digital image 
matrix of 512 x 480 pixels. The complete image 305 is 

20 used as an input to various other subroutines in the 
enrollment process which will be described below. 

Returning to Fig. 3A, the next step in the 
enrollment process is the binarization process 310. 
This process is illustrated in detail in Fig. 5. The 

25 function of the binarization process 310 is to convert 
8-bit 256 gray scale pixels of the input matrix into a 
two-level color output. As shown in Fig. 5, step 310 
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receives as the input matrix the complete image 305. 
At step 3102, a center image is taken from the complete 
image. The coordinates of the upper left hand corner 
of the center matrix is defined as coordinates 128 x 
5 120 and the coordinates of the bottom right hand corner 
of the center matrix is defined as coordinates 384 x 
360. This central image is then binarized. This 
process results in an image where each pixel is either 
black or white, depending on whether a pixel exceeds or 
10 falls below a preset threshold. 

As noted, the coordinates are chosen for the 
preferred embodiment to focus on the center of the 
image. However, if the object does not have 
distinguishing features in its central area, different 
15 coordinates can be used. This output image is then 

made available to the targeting procedure 32 0 shown in 
Fig. 3A of the enrollment process. 

An alternative embodiment for the targeting 
process is also shown in Fig. 28. The targeting 
20 procedure 320 is shown in more detail in Fig. 6. The 
purpose of the targeting procedure is to find a 
distinguishing feature in the object in order to detect 
the presence of the object and determine the location 
of the object in the image matrix. In the preferred 
25 embodiment, the distinguishing features looked at are 
the two irises of the eyes. As shown in Fig. 6, the 
input to the targeting function 320 is the binarized 
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image 3104 which is then processed by the labeling 
function 3202. In the preferred embodiment, the 
labeling function 3202 locates and labels all areas 
that exhibit similar characteristics as irises. In 
other words, the threshold set for the binarization 
process 3102 is set to filter out gray scale levels 
that are not relevant. Thus, the gray scale color 
typically associated with irises can be used as the 
indicator for the threshold. The output of the 
labeling process 3202 comprises the object labels 3204. 



Next, the coordinate calculation process 3206 
is activated. In this step, each labeled object 
produced at step 3204 has the XY coordinates calculated 
for placement in the complete image matrix 305. This 
provides a geometric center of each object that was 
labeled in the previous step. 

Thus, in the first embodiment, in step 3206 the 
irises are located and are distinguished by their 
contrast with the surrounding area. In addition, other 
contrasting areas may also be labeled. These 
contrasting areas, are for example in this exemplified 
application, nostrils or lips. 

The next step in the targeting process 320 of 
Fig. 6 is the coordinate validation step 3208. Step 
3208 involves looking at the XY coordinates of a pair" 
of objects and then determining whether their absolute 
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and relative locations are valid. The validation step 
3208 assumes that labeled objects, such as the irises, 
are appropriately positioned on a face. For example, 
the eyes cannot be on top of each other and must fall 
5 within acceptable distances from each other. 

Therefore, the validate coordinate step 3208 function 
determines those pairs of labeled objects that can 
possibly be irises. 

Specifically, the calculations for iris 

10 targeting consists of comparing the XY coordinates for 
each iris to determine if they are within a preset 
distance apart and on approximately the same horizontal 
axis. In addition, the difference in the X coordinates 
are measured and compared to a prestored value to make 

15 sure that the irises are located at certain specific 
locations. In the preferred embodiment, the 
coordinates Yl and Y2 represent the horizontal 
coordinates, and XI and X2 represent the vertical 
coordinates. Thus, Y2 and X2 in the preferred 

20 embodiment represent the left iris coordinate, and Yl 
and XI, the right iris. 

A first calculation determines if Y2 is greater 
than Yl . In the second calculation, the result of Y2 
minus Yl should be greater than a value of 40 pixels. 

25 The third calculation determines the absolute value of 
XI minus X2 . In the preferred embodiment, that value 
should be less than 16 pixels. If all three equations 
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are met, then at step 3208 the object's pair of irises 
is confirmed, and processing passes to step 3216. 

However, if no valid targets are found (failure 
to pass at least one of the above three equations) an 
5 output message is sent at step 3212 to monitor 50 (Fig. 
1) stating that the process has been unable to target 
the eyes. A new image is acquired again and 
reprocessed, beginning at step 302. 

The next step 3216 is to validate the object. 

10 This step compares the spots with the eye template to 

determine whether a cross correlation coefficient fits. 
If so, it confirms that the system successfully 
targeted the eyes. 

Any one input to the validate object step 3216 

15 is determined at step 315. This input is an average 
eye template value, which is an average of the iris 
position on the face across a wide population. The 
other input, determined at step 305, which was 
discussed previously, is the complete image. As 

20 noted, the complete image is a reference standardized 

noise-free image matrix of 512 x 480 pixels, 8-bit gray 
scale. The validate object step 3216 performs a gray 
scale correlation using the complete image 305 and the 
average eye template 315 and the valid object XY 

25 coordinates. This complete image is compared to the 
average eye template at the valid XY coordinates. " If 
the maximum correlation is above a preset threshold, 
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the object is identified as an eye. The correlation 
coefficient of two areas {A^} and {b^} is calculated 
as : 

P=Z„A 9 b 9 l fi 9 A*L 9 b* 

5 

where A Aj and b^ are pixels of the two areas. 
The threshold of correlation is 0.9. 
The outputs of this comparison are two "valid" 
iris values with the associated XY coordinates in the 
10 complete image matrix 305. The outputted values are 
provided at 3218. 

Returning to Fig. 3A, following the targeting 
process 320, the system then retrieves the calculated 
unit distance/center point by initiating the process 
15 set forth at step 325. A detailed flow chart of this 
process is shown in Fig. 7. 

Referring to Fig. 7, the calculate unit 
distance and center point routine 325 establishes a 
unit distance for the center point in the image based 
2o on the coordinates of the iris provided from the 

targeting step 32 0. The unit distance (UD) equals 
((X1-X2) exp 2 + (Y1-Y2) exp 2) exp ^. The center 
point (CD) equals, for the X coordinate, CX = (XI + 
X2)/2 and for the Y coordinated CY = (XI + Yl)/2. 
25 The next step of the enrollment process shown 

in Fig. 3A, is to define an area of interest at step 



.9705566A1J_> 



WO 97/05566 * 4 PCT/US96/1 1996 

330. The area of interest procedure 330 is shown in 
detail in the flow chart diagram of Figure 8. 

Referring to Fig. 8, the function of step 3301 
is to define the areas of interest on the object in 
5 relation to the unit distance (UD) and center point 
values (CX and CY) . The areas of interest are 
predetermined depending on the object to be identified. 
In the preferred embodiment, eight areas of interest 
have been selected. These areas of interest are a 

10 one-dimensional horizontal bar on the forehead, a one 

dimensional vertical bar over the center of the face, a 
two-dimensional right and left eye section, a 
two-dimensional right and left eyebrow section, and a 
two-dimensional right and left cheek section. 

15 Essentially, the areas of interest for a face in the 
preferred embodiment are dissected into two 
one-dimensional areas and six two-dimensional areas of 
interest (see Fig. 20) . 

Returning to Fig. 3A, once the areas of 

20 interest are determined, the next step is to normalize 
the areas of interest at step 335. Step 335, shown in 
detail in Figure 9, resizes the area of interest to a 
standard pixel size. In the preferred embodiment, the 
standard pixel size for the one-dimensional pixel area 

25 of interest is 8 x 64 pixels. For the two-dimensional 
areas of interest, the standard pixel size is 64 x 64 
pixels. The purpose of this normalization procedure 
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step is to standardize the input to the transform 
procedures . 

Step 34 0 shown in Fig. 3A then performs several 
transforms, each of which is applicable to particular 
5 areas of interest. One of these transform processes is 
step 342. Specifically, step 342 applies transforms to 
the one- dimensional pixel arrays (representing the 
eight areas of interest) outputted from the resized 
areas of interest step. 

10 As seen in Fig. 10, eight fast-fourier 

transforms (FFTs) , as known in the art, are performed 
at step 3420 on the one-dimensional pixel arrays. The 
results of these transforms are averaged at step 3422 
into a 1 X 64 vector array representing the spatial 

15 distribution of that area of interest. Processing then 
returns to Fig. 3A. 

Another transform is performed at step 344 
specifically applied to the dimensional areas of 
interest. The transform used is a discrete cosine 

20 transform (DOT) , as known in the art (see Fig. 11). 

Referring to Fig. 11, each 64 x 64 pixel array 
is divided into 64 separate pixel arrays of 8 x 8 
pixels at step 3440. Then each 8x8 pixel array is 
compressed using the DOT at step 3442. The output of 

25 the DOT for each 8x8 pixel array is a transformed 

array with the most significant cell in the upper left 
hand corner. Using all the 8x8 transformed arrays, 
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ten 1 x 64 vectors arrays of the most significant cells 
are then created at step 3444. Other techniques can be 
employed, such as edge detection, Kohonen f s and/or 
geometrical analysis. Step 346 in Fig. 3A, depicts 
> these other alternative transforms which can be used to 
compress and analyze identified areas of interest. 

For example, the most significant cell of each 
of the 64 transformed arrays comprises the first 1 x 64 
vector array, the second most significant cells 
comprises the second 1 x 64 array, and so on. The 
result is that each 64 x 64 pixel area of interest is 
transformed into ten 1 x 64 vector arrays of the most 
significant transformed cells. These arrays are then 
sent to the coding routine 350. 
15 As an alternative to the above, each layer can 

be binarized, so that if each cell's coefficient is 
greater than zero, then the value for that cell is 
equal to one. If that cell's value is less than zero, 
then it's binarized value is equal to zero. As a 
20 consequence, relatively few bytes for multiple layers 
are necessary. For example, if each layer is 8x16 
bytes, then the binarization will create and an 8x16 
bit layer. For a 6-layer image, for example, 96 8-bit 
bytes (6x16) will be created for the captured image. 
25 Thus, under this alternative binarization step a very 
small amount of memory is necessary for the UD/CP 
values . 



OCID: <WO 9705566A1_I_> 



WO 97/05566 z/ PCT/US96/11996 

Referring now to Fig. 3B, the coding routine 
350 is shown. Fig. 12 sets forth routine 350 in more 
detail. Referring to Fig. 12, input to the coding 
routine array are the sixty-two, 1 x 64 vector arrays 
5 produced by the transform routine at step 340 (Fig. 

3A) . For each one-dimensional area of interest, one 1 
x 64 vector array is inputted. For each 
two-dimensional area of interest, ten 1 x 64 vector 
arrays are inputted. Therefore, in the preferred 
10 embodiment, 62 1 x 64 vector arrays are inputted to 
routine 350. 

The other input 355 is the eigenspace. The use 
of eigenspaces is well-known in the art as a method for 
determining the characteristics of an individual 

15 observation to a sample of the general population. See, 
for. example, Kirby, M and Sirovich, L. Application for 
Karhunen-Loeve Procedure for the Characterization of 
Human Faces , IEEE Trans. Patt. Anal. Machine Intell., 
Vol. 12, pp. 103-108, 1990; Turk, M. and Pentland, A., 

20 Eigenfaces for Recognition Journal and Cognitive 

Neurosciences , Vol. 3, pp. 71-86, 1991; Gonzalez, R.C. 
and Woods, R.E., Digital Image Processing , 
Addison-Wesley, 1992 . 

The first coding step 3502 calculates residuals 

25 of the vectors. The residuals are the differences 
between the sixty- two vectors and the mean vectors 
estimated for a general population. Next, these 
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residuals are projected into their sixty- two separate 
eigenspaces, permitting one per parameter. The result 
of this process provides the two most significant 
coordinates, per parameter, in their respective 
5 eigenspaces. In total, 124 coordinates are calculated. 
Process step 3504 is repeated several times to insure a 
statistically appropriate sampling of the enrollment 
images calculates the mean and standard deviation of 
the 124 parameters coordinates generated at step 3504. 

10 Step 3508 then evaluates the coordinates with the 

smallest standard deviation and highest coefficient 
with the average of the population. Based on those 
criteria, the coordinates and their respective weights 
are then passed to the encryption process 370. 

15 The encryption routine 370 of Fig. 3B is shown 

in detail in the flow chart of Fig. 13. Such a routine 
is well known in the art. For example, the encryption 
algorithm shown at step 3702 determines usable 
parameters according to encryption criteria 3704 which 

20 are related to the mean and the standard deviation of 
the parameter coordinates. The result is the 
encryption key and verification data which are written 
at step 3706 onto the portable storage 70. However, a 
code or any other well known technique in .the art of 

25 recording information can be used. Therefore, the card 
70 contains the coded information pertaining to the 
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object that was enrolled. The enrollment process is 
now complete. 

Fig. 14A & Fig. 14B show an overview of the 
verification process using the verification station 
.5 hardware shown in Fig. 2. Most of the procedures in 

the verification process are similar to the procedures 
previously discussed regarding the enrollment process. 
Thus, a detailed explanation is reserved for those 
processes that differ. A detailed description of the 

10 verification steps are set forth in Figs. 15-19. 

A prerequisite to the verification process 400 
is for the enrollment process to be repeated, up to a 
certain point. Specifically, in the preferred 
embodiment, the person that needs to be verified would 

15 go through step 302 (preprocessing) through step 350. 
The output of step 350 in the verification process 
provides parameter values corresponding to the images 
of the person or object to be verified. At the same 
time, card 70, which contains the data from the 

20 enrollment process, is inserted into a reader device 

which then decrypts the magnetic data, yielding process 
control data (Fig. 16 step 410) and parameter values 
that correspond to each area of interest (Fig. 17, 
process 420) . The process control data instructs the 

25 machine on how to accomplish verification. The 

parameters- values determine what is to be verified. The 
parameter values are compared to the output of the 
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coding step 350 in the verification process (Fig. 18 
process step 430) . Statistical analyses, as is 
generally known in the art, is used to accomplish 
verification. Other methods, however, such as 
5 ruled-based identification decision processes or fuzzy 
logic systems, may be used in addition to the 
straight-forward statistical analyses. Furthermore, a 
degree of correlation between the two values can be 
varied depending on the degree of sophistication of the 

10 verification technique desired. If at the decision 

making step (Fig. 19 process 440) the parameters of the 
card match the parameters of the photograph image, 
verification has been achieved and an output is made to 
the external control unit process 450. 

15 One verification methodology for example can 

rely on increasing the Hamming distance between the 
enrolled image and the image to be verified. In this 
instance, the image vector stored in the card, or other 
non-volatile storage media is lined up, bit-by-bit to 

20 the generated image vector. The bits for each vector 

are compared. For different bits, a value of "1" is 

generated, for identical bits, a "0". A Hamming 

distance is then generated, as follows: 

where n = total number of bits in the vector 

25 where N = total length of vecot 

(normalized) Hamming Distance (HD) = n 

N 
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The HD value can be used as a threshold value, 
from which system sensitivity can be varied. If the 
values are set at around, for example .23, other 
sensitivities for retest, for example, or for reject 
5 can also be set. If for example, accept is .23 (HD) , 
retest is .24 - .74 and reject is .75 or greater, then 
it is possible that over time the retests will migrate 
to either direction (i.e., accept, reject). 

Figs. 21-27 respectively illustrate a 

10 micro-camera assembly and LED lighting apparatus which 
provide numerous operational advantages both to the 
various embodiment of the invention, as well as to any 
other known image enrollment/recognition systems. 

In particular, Fig. 21 illustrates a front-view 

15 of an array of light emitting diodes ("LEDs") 2102 
located along the same plane of a plate 2104. The 
arrangement of the LED's has a specific size, and 
intensity, to optimize lighting intensity of the target 
and capture of the image. In this diagram, a 

20 configuration is shown for maximizing iris capture. 
The LED's are designed to light the iris at a lower 
visible infra-red spectrum, rather than at the "heat 
detecting" spectrum. The spectrum tends to have a wave 
length of approximately 880nm, although other low 

25 visible spectra are considered applicable. Infra-red 
spectrum light has been found to be optimal, since 
flash lighting is distracting, if not painful. 
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Moreover, infra-red, at low levels, standardizes facial 
fill so that all features of the face are equally 
exposed. A further advantage to low level infra-red is 
that, when combined with an appropriate filter, ambient 
5 light is cut out altogether. Consequently, image 

wash-out is avoided. Finally, the higher spectrum heat 
detecting level has been found to be less accurate in 
measuring biometric characteristics. 

As shown in Fig. 21, an array of nine LED's 

10 2106 are arranged in a square that is angled a 45° 
relative to the horizontal axis 2108 of plate 2104. 

Fig. 22 is a transparent perspective view of 
the microcamera device 2200 incorporating the 
aforedescribed LED array 2100. The device includes 

15 four basic elements: a micro-camera lens 2202, a 

microcamera circuit board 2204, the aforedescribed IR 
LED array 2100 and the LED circuit board 2206. These 
four elements are contained in a housing 2208. 

The housing 2208 is designed so that the lens 

20 and LED array are held in a potting material in order 

that the microcamera unit may be contained, and sealed. 
As a consequence, the microcamera can be used in 
underwater applications without substantial water 
leakage to circuit boards 2204 and 2205. The potting 

25 has a sufficient circumferential clearance around the 
lens element 2202 in order to allow the lens to freely 
rotate . 
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The top surface of the housing 2210 contains a 
recess 2210 the top surface of which is co-planar with 
the top surfaces of the lens 2202 and LEDs 2102 . 
Further, a pair of flanges are arranged parallel to 

5 each other and the longitudinal axis of housing 2208 so 
that a flat filter element (not shown) that is sized to 
fit between the flanges can slide across the top 
surface 2210 and be held in place by the flanges. The 
filter comprises a sheet of mirrored glass or plastic 

0 that is near infra-red. The filter is thus able to cut 
off the visible light spectrum. 

The mini camera housing includes a 
communications port 2220 which provides the image 
output to an external frame grabber. The port 220 also 

5 connects to an external power supply for the 

mini-camera. The port 220 may use any optimal 
wiring-confirmation which in this embodiment a 9 pin 
DIN that can connect to the I/O port of a PC. 

In a second embodiment, the camera device 220 

0 has no potting. However in this embodiment, a wall 
2222 would be placed between the lens 22 02 and LED's 
2102 to avoid direct reflection on the filter by the 
LED's. 

The camera lens 2202 and camera circuit 2204 
5 are manufactured by Sony Corp. as a 1/3 CCD, 12 VDC 
Board Camera, Model No. UL-BC4 60/T8. 
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Referring now to Fig. 23, a schematic circuit 
board layout 2300 is shown for the LED array 2100 
(elements D1-D8) . In this arrangement, the lighting 
for diodes D1-D8 is continuous but has a quick "on-off" 
5 to cover a video view. This on-off cycle is 

approximately l/30th of a second. Moreover, the flash 
component of the video view period is l/7000 tn of a 
second. Since the period of lighting is so brief, the 
flash and the lighting exposure render sensitivity to 

10 movement of the subject pratically irrelevant. Flash 
nonetheless is essential since in security 
applications, movement of the subject occurs 
frequently. However, the flash can be changed to a 
continuous lighting mode, if desired. 

15 Each of the IR LED ' s is a focused beam diode, 

which improves efficiency and also reduces power 
consumption. As noted previously, pin connections 2301 
are adapted to connect directly into a personal 
computer I/O port. 

20 Fig. 24 is an illustration of the circuitry 

supporting the camera electronics 2400. A constant 
power source of about 100 million amps is provided. A 
12 volt power supply is used along with a 5V control 
power supply. 

25 As a consequence of the arrangement of FIGS. 

21-24 a micro camera arrangement for image capture is 
created whereby the lighting is located below the 
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camera. Moreover, the position at which the lighting 
is below such camera is critical, since a subject 
farther than 3 feet away from the lens will not be 
captured. Placement of the camera is also sensitive 
5 since direct sunlight, incandescent or halogen light 
will wash out features. Thus any direct light to the 
camera is problematic. 

Figures 25a-c are different views (front, side, 
and perspective) of a housing designed to contain the 

10 camera-LED unit. A recess 2502 is shown in the unit, 
through which the entire housing 2200 can be inserted. 
The modular plug 2220 (Fig. 22) would also be connected 
through cable 2504 (Fig. 25c) to the PC I/O port (not 
shown). In the side view of Fig. 25(b), the housing 

15 2504 includes a stand 2506 which pivots about axle 2508 
in the direction of arrow 2510. As a consequence, the 
camera can be supported in a substantially upright 
position (Fig. 25(c)) when placed with the stand in an 
extended position on a horizontal surface. 

20 Figs. 2 6a-c show a second embodiment of the 

mini-camera housing 2 600. As shown, the housing 
includes a stand 2602, which in a closed position (as 
shown in Figs. 26a, 26b), completely covers the camera 
lens and LED f s. When fully opened, however, which is 

25 accomplished by rotating the stand 2602 about axis 

2604, the camera, and LED light unit "are fully exposed, 
and is also supported upright by stand 2 602 (Fig. 25c) . 
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Finally, Figs. 27a-c are views of a third 
embodiment of the minicamera housing 2700. In this 
embodiment, a stand 2702 is partially cut away, to 
expose the camera lens only. However, when the stand 
5 is opened (Fig. 27(c)) the LED array and the camera 
2202 are both exposed for use. 

Fig. 28 illustrates a second or alternative 
embodiment for the targeting process set forth in Fig. 
6 of this invention. The advantage of the alternative 
10 technique is that it allows targeting without reference 
to fixed areas, by dynamically finding the image 
centers. 

The process, 2800 begins at step 2802 where a 
desired area is isolated and captured by the camera. A 

15 histogram for this captured area is then computed by 
the computer 40. The computer then • dynamically 
determines thresholds by calculating desired threshold 
barriers that are preprogrammed into the computer 40. 
For example, high and low rejects can be set to be 

20 above the lowest 5% and below the highest 5%, and high 
and low thresholds between the bottom 45% and below the 
top 45%. As a consequence, when the threshold is 
compared to the histogram at step 2 808, a 10% middle 
portion of the histogram can be defined reflecting 

25 particular gray-scale characteristics. 

The below, between," and above threshold values 
are then binarized at binarization step 2810 as shown 
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in 28(c). The first binarization step is the threshold 
comparison itself, which sets values as follows: 
where: HT = high threshold 
LT = low threshold 
5 any value > HT=1 

any value < HT=0 
any value < LT=1 
any value > LT=0 
the binarized values are then compared, and a majority 
10 fill, as shown in Fig. 28(d) occurs, which represents the 
true binarized area of the targeted object 2812. 

The targeted area is then geometrically tested 
at step 2814 on two candidate points based on preset 
values which define appropriate quadrants. The points 
15 x>irYi and x 2 ,y 2 can be isolated based on preset template 
values. For example, if iris targeting is desired, eye 
templates can be set so that |X 2 -X 1 |>40 and |Y 2 -Y 1 |<10. 
Once correlation of the templates has occurred at step 
2816, the image is divided, and the lower portions of 
20 the image are dropped-out, leaving only the 
side-by-side quadrants . 

In the event that targeting cannot locate the 
desired targets with the appropriate threshold 
characteristics, the system can keep expanding the 
25 image area in order to locate the desired points. 
Specifically an iteration loop can take three (3) 
images, binarize those values, average those binarized 
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value and store the averaged value in the portable 
memory. As a result of this iteration process shown at 
steps 2822 and 2824, a high percentage of accuracy is 
achieved dynamically. 

The above description and drawings are only 
illustrative of preferred embodiments which achieve the 
objects, features and advantages of the present 
invention, and it is not intended that the present 
invention be limited thereto. Any modifications of the 
present invention which comes within the spirit and 
scope of the following claims is considered part of the 
present invention . 



15 



20 



25 



9705566A1J_> 



WO 97/05566 



39 



PCT/US96/11996 



We claim : 

1. A system and method for obtaining, 
digitizing, and recording selected data sets of 
separate and independent component element image 
5 signals, sensed by photographic or electronic means, 

from a person or object, for subsequent comparison with 
similarly obtained component elements image signals of 
a person or object whereby the degree of similarity 
between the original and the subsequent data sets may 
10 be determined. 

2. A method, as in claim 1, whereby data sets 
of sensed images may be recorded on portable media, 
such as magnetic stripe cards, magnetic discs, printed 
bar codes on paper or similar materials, semi-conductor 

15 devices such as smart cards, or in data bases. 

3. A method, as in claim 1, of selecting 
separate and independent component elements of persons 
or objects whereby the selected components provide 
image signals for recording quantitative structure or 

20 array of numbers that permit statistical identification 
of the individual person or object from among data sets 
previously obtained from the same person or similar 
objects . 

4. A method, as in claim 3, whereby the image 
25 signal data sets of persons or objects are measured so 

as to permit the determination of a quantitative degree 
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to which the objects or persons are similar to those 
originally measured. 

5. A method as, in claim 3, whereby it may be 
determined whether component image signals obtained 

5 from a person or object, at any time subsequent to the 
recording of a set of such image signals from the 
person or object, differ, or do not differ, from the 
recorded component images by any, or by more than or 
less than, preselected quantitative values, permitting 
10 the system to indicate that the signals may be 

quantitatively, or qualitatively, verified, or not 
verified, as being from the recorded signals of the 
person or object. 

6. A system, as in claim 1, whereby 

15 quantitative measures may be selected for inclusion in 
the digital data system for allowing the determination 
of chosen levels of difference between subsequently 
obtained component image signals and verification of a 
such signals as being, or not being, from the recorded 

^0 signals of a specific person or object. 



25 



.9705566A1J_> 



WO 97/05566 



PCT/US96/11996 



C:\1 1_SYS\ABCVPATENT\FIG001 AF3 
Tuesday, June 6, 1995 
9:40 AM 



SUBJECT 



30 



LIGHTING 
standard light 

STILL VIDEO 

multiple 

cameras 



100 



42 



DIGITIZER 
frame grabber 



STORAGE 



FIG. 1 ENROLLMENT STATION 



40 



COMPUTER 



44 

7 



CPU 



MONITOR 



46 

J 



MEMORY 



KEYBOARD 



T 

50 



10 



20 



22 



-7 — 

"72. 



^ OUTPUT DEVICE 



60 



70 



"Portable 
Memory 



Page 1 



1/30 

CID: <WO 9705566A1_L> 



WO 97/05566 



PCT/US96/11996 



C:\1 1_SYS\ABC\PATENT\FIG002.AF3 
Tuesday, June 6, 1995 
9:40 AM ; 



30 



LIGHTING 
standard light 



SUBJECT 



STILL VIDEO 

multiple 

cameras 



200 



FIG. 2 VERIFICATION STATION 



42 



. DIGITIZER 
: frame grabber 



STORAGE 



40 



COMPUTER 

44 

— C 

CPU 



MONITOR 



46 



MEMORY 



KEYBOARD 



210 



20 



48 



V 

50 



52 



220 



— j INPUT DEVICE 



230 




Page 1 



2/30 



XJCID: <WO 9705568A1 J_> 



WO 97/05S66 



PCT/US96/11996 



C:\11_SYS\ABC\PATEISIT\ENROLL\ENROL004 AF3 Fi 9- 3A ENROLLMENT 



TM 



Thursday, March 2, 1995 
12:43 PM 

SENSOR INPUT 

SUCH AS AN IMAGE 
FROM A CAMERA 



SETUP DATA 

SETUP OR 

CLASSIFICATION OF THE — 
ENROLLMENT 
PROCEDURE 




. PRFPROr.FSSINfi 

DIGITIZING SENSOR rNPUT 



-BINARIZE- 



BfNARIZE A INVERT IMAGE 
CENTRE 



£*w binarized image centre 

' I 

TARfiFTING 



r 
r 



ONE-ON-ONF 

ENROLLMENT 



300 



305 



320 



. COMPLETE IMAGE 

STANDARDIZED MOtSE 
FREE IMAGE MATRIX 512 X 
480 8 BIT 256 GRAY SCALE 



LOCATING IRISES 



eye template 



315 



r 



AVERAGE EYE 

(DATA STORED 



2 valid irises with XY coordinates 

* ^-325 



-CALCULATE UD/CP 

C ALC. UNIT DISTANCE AND 
CENTRE PIONT 



3 



valid UD&CP 
▼ 



OFFINF A o I 



330 



AREA of INT EREST 
SEPARATION 



t ) 



Ao.I. 




• c) 

setup data: 
color thresholds, 
face area & 
targeting validation 



setup data: 
/ selected A.o.1. 



-RESIZE A.O.I.- 



335 



NORMALIZE Ao.L 



J 



* J ID 8X64 / 2D 64X64 pixels 



340 



-342. 



TRANSFORMS 



EITHER ID FFT ■ — 

T11 . T12 TIN \ 

I I 



OR2DDCT 



44. 



346^ 



OR PATC ft RECOGNITION 



SNITION 



V 



T21 y T22 - T2N ^ 



Ti1 



Ti2 V TiN 

A 

^ 0m m* * 



) transform output ID one 1 x64 / 2D ten 1 x64 arrays 



CONTDsTUE NEXT PAGE 



3/30 



OCID: <WO 9705566A1_I_> 



WO 97/05566 



PCT/US96/11996 



transform output ID one 1x64 / 2D ten 1x64 arrays 



EIGENSPACE 

< DATA STORE) ^35 5 



GENERAL SAMPLE FOR 
EIGENSPACE 



r 



350 



COOING 



CLASSIFYING OUTPUT 
- A.o.1. TRANSFORMS 



) 



Fig. 3B ENROLLMENT 

r r 



i 



CARD 

IV SETUP DATA 

2.) VERIFICATION DATA 

CP ARAM. KEY A VALUE) 



verification data: 
(param. key & value) 



-vectors- 



DATALOG 

I DATA USED FOR 
BUILDING NEW EIGEN- 
SPACES 



parameters value 



36( 

Y 



ENCRYPT 



370 



970556eA1J_> 



4/30 



WO 97/05566 



PCT/US96/11996 



CA1 1_SYS\ABC\PATENTVENROLL\EPREP004.AF3 Rg 4 PREPROCESSING 

Thursday, March 2, 1995 
12:44 PM 



22 



SENSOR INPUT 

I SUCH AS AN IMAGE 
FROM A CAMERA 



DIGITIZE 



TM 



ONE-ON-ONE 



PREPROCESSING 
* output: a standardized noise free 
digital image matrix 512 X 480 8 bit 
256 gray scale. 

- function: digitizing, filtering and 
enhancing the image from the camera 
to the enrollment process. 
Preprocessing is external to the 
enrollment process. It is solely 

: dependent on the input device and the 
frame grabber. Ail preprocessing takes 
place on the frame grabber. 

- input: analogue signal from the 
camera. 



3002 



FIL TER 



3^2 



3004 



ENHANCE 



3006 



COMPLETE IMAGE 

* COMPLETE NOISE FREE IMAGE 
MATRIX 512 X 480 8 BIT 256 
GRAY SCALE 



I 



BINARIZE 



310 



5/30 



XID: <WO 9705566A1J_> 



WO 97/05566 PCT/US96/1 1 996 



C:\11.SYSVABC\PATEN7\ENROLL\EBINA004 AF3 pjg # 5 BINARIZE 
Thursday, March 2, 1995 
12:43 PM 



TM 



302 



PREPROCESSING 



ONE-ON-ONE 310 

BINARIZE ~ 
- output: a 256 X 240 matrix of binarized and 
. inverted centre of image. 
; - function: copies the centre of the image and 
| binarizes this part. The function leaves the 

• complete image untouched for later processing. 

• -input: standardized noise free image matrix. 



COMPLETE IMAGE 

COMPLETE NOISE FREE 
' IMAGE MATRIX 512 X 480 8 
BIT 256 ORAY SCALE 



305 



T 



310 



( BINARIZE 

I BLNAKlZJb AND BTVERTE — 
IMAGE CENTRE TO A 256 X 
240 MATRIX COORD: 
UUf 128 X 120 
BRH 384 X360 



02. 



\ 



3J0f 



BINARIZEO IMAGE 

. 256 X 240 MATRIX OF 
IMAGE CENTRE 1 BIT 

black at worrE 

AREA FROM COMPLETE 
IMAGE COORD: 
-ULH 128X120 



BRH 384X360 



f 



i 



& 

TARGETING I 



tOCID: <WO 970SS68A1 



6/30 



WO 97/05566 



PCT/US96/11996 



C:\1 1_SYS\ABC\PATENT\ENROLL\ETARG004.AF3 
Thursday, March 2, 1995 
12:44 PM 



Fig. 6 TARGETING 



BINARIZE 



310+ 



pio 

" - nntr 



BINARIZED IMAGE 

256X 240 MATRIX OF 
IMAGE CENTRE 
ULH 1 28X 1 20 BRH 384X360 
I BIT BLACK & WIHTE 



3202 



LABELLING 



OBJECT LABELING: 
- FINDING & LABELING HIGH 
ENERGY SPOTS IN THE IMAGE 



3206 



CALC. COORD 

FOR EACH LABELED OBJECT 
THE XY COORDINATES ARE 
CALCULATED IN LABEL MATRIX i 



320 



3208 



K * 



COORD VALID 

XY COORDINATES ARE 

CALCULATED FOR COMPLETE 
IMAGE, VALIDATED 
* I. Y2> Yl . 

2. Y2-YI >^ 

3. !X1-X2|<I6 



all valid high energy spots with 
valid XY coordinates possibly irises 



TM 



ONE-ON-ONE 



320 



TARGETING 

- output: 2 valid irises with their XY 
coordinates. 

- function: finding the face in the image by 
locating the irises. 

- input: I. binarized and inverted centre of 
image 2. the complete image for 
determination of the irises. 

3. average eye template 



OBJECT LABEL 

SAME AREA & SIZE AS 
BINARIZED IMAGE FILE 256 X 
240 MATRIX OF IMAGE CENTRE 
ULH 128X120 BRH 384X360 
GRAY SCALE DEPENDS ON 

„ NUMBER OF OBJECTS IN IMAGE. 

' ONE SHADE OF GRAY = ONE 
OBJECT 



3204 



305 



COMPLETE IMAGE 

STANDARDIZED NOISE FREE 
IMAGE MATRIX 512X480 8 BIT 
256 GRAY SCALE 



315 



AVERAGE EYE 

2 AVERAGE EYE TEMPLATE ONE 
LEFT EYE ONE RIGHT EYE. 



\Mab\e. 



OBJECT VALID 



LUMPAKk VALID SPOTS WITH 
EYE TEMPLATE IF 
CROSSCORRELATION 
COEFFICIENT FITS = THIS IS AN 
■ EYE 




3218 

2 valid irises with 



CALC. UD/CP 



their XY coordinates 



f 



3216 



325 



7/30 



OCID: <WO 9705566A1 J_> 



WO 97/05566 



PCT/US96/11996 



CM 1_SYS\ABC\PATENT\ENROLL\EUDCP004.AF3 
Thursday, March 2, 1995 
12:44 PM 



TARGETING 



Fig. 7 CALCULATE UNIT DISTANCE 
& CENTRE PIONT ™ 

ONE-ON-ONE 325 

_ CALC UD/CP ) 

- output: 1 UNIT DISTANCE = UD 
; 2 CENTRE POINT = CP 

, * function: establishes UD and CP based 
on the coordinates of the irises. 

- input: 2 valid irises with their XY 
coordinates. 



325 < 



2 valid irises with coordinates 



3250 



■ CALC . UD/CP 

UNIT DISTANCE = 



: (,(Xl-X2Vxp2 + (YI-Y2)ocp2)expl/2 

■ centre piont = 

CX = (X!+X2V2 
CY*<Xl-Yiy2 



Unit Distance / Centre Point 



r 



DEFINE A.O.I 



330 




8/30 



) 9705568A1_I_> 



WO 97/05566 



PCT/US96/11996 



CA1 1 _S YS\ABC\P ATENTVENRO LLAEDAI P004. AF3 
Thursday, March 2, 1995 
12:43 PM 



Fig. 8 DEFINE AREA of INTREST 

TM 

ONE-ON-ONE 330 



DEFINE AO I 

- output: the AREAS of INTEREST as 
calculated for the individual, according to 
the size of the face and the distance to the 

: camera. The A o I s are relative to the 
dimension of the unit distance and the 

'. location of the centre piont. 

: - function: define the AREAS of 
rNTEREST on the object in relation to the 
unit distance an centre piont 

- input: 1. UNIT DISTANCE = UD 
2. CENTRE POINT = CP 



325 



CALC. UD / CP 



unit distance / centre point 



DEFINE A.O.I. 

i ID HORZ. BAR ON FOREHEAD. I D VERT. 
BAR OVER CENTRE, 2D RIGHT & LEFT EYE 
SECTION. 2D RIGHT &. LEFT EYEBROW 
: SECTION. 2D RIGHT & LEFT CHEEK 
3<*V SECTION 



separated AREAS of INTEREST 



335 



RESIZE A.o.1. 



9/30 



DC ID: <WO 9705566A1 J_> 



WO 97/05566 



PCT/US96/11996 



C:\1 1_SYS\ABC\PATENTVENROLL\ERAIP004 AF3 
Thursday, March 2, 1995 
12:45 PM 



Fig. 9 RESIZE AREA of INTREST 



33< 



DEFINE A.o.1. 



separated AREAS of INTEREST 



ONE-ON-ONE 



TM 



335 



RESIZE A.o.1 

- output: AREAS of INTEREST 
normalized to a standard pixel size. 

- function: resizes AREAS of INTEREST 
to a standard pixel size. 

ID to 1x64 pixels. 
2D to 64x64 pixels. 

- input- the AREAS of INTEREST as 
calculated for the individual according to 
the size of the face and the distance to the 
camera. The A.o.Ls are relative to the 
dimension of the unit distance and the 
location of the centre point. 



335 



RESIZE A.o.1. 



335 



'. RESIZE AREAS OF INTEREST 
I D TO I XW PIXELS 
2D TO 64X6-1 PIXELS 



r 



pixel normalized A.o.1. 

i 

340 

TRANSFORMS 



10/30 



) 9705566A1J_> 



WO 97/05566 



PCTYUS96/11996 



C:\11_SYS\ABC\PATENTVENROLL\ETRAN104.AF3 fjg 1D TRANSFORM 

Thursday, March 2, 1995 
12:46 PM 



335 



RESIZE A ol. 



8x64 pixels A^oA. 



ONE-ON-ONE 



TM 



1 D TRANSFORM 

- output: standard ID 1x64 array 

- function: transform the pixe! input 
through a FFT into a spatial distribution 
array. 

- input: AREAS of INTEREST 
normalized to a standard pixel size. 



3420 



FFT 



FOURIER 
TRANSFORMS 



spatial distribution 



3422 



342 



MAKF ARRAY' 



1x64 array 



CODING 




350 



11/30 



XID: <WO 9705566A1_I_> 



WO 97/05566 



PCT/US96/11996 



C:\1 1_SYS\ABCVPATENTVENROLLVETRAN204.AF3 
Thursday, March 2, 1995 
12:46 PM 



Fig. 11 2D TRANSFORM 



REZISEA.ol : 



j? 1 



64x64 pixels A.o.1. 



niVinF in fiXfl 



DIVIDE INPUT INTO 64 
; CELLS OF 8X8 PIXEL 



3440 



TM 



ONE-ON-ONE 



2D TRANSFORM 
• - output: standard 2D 10x64 array. 
= - funci ion: transform the pixel input 
through a 64 DCTs into a spatial 
; distribution array of the most significant 
; cells. 

\- input: AREAS of INTEREST 
; normalized to a standard pixel size. 



^344 



344 



64 8X8 pixel areas 



•: dci 

64 DISCRETE COSINE 
I TRANSFORMS PER A.o.1. 



3442 



spatial distribution 



MAKP APPAV 

: 10 MOST SIGNIFICANT 

! CELLS STORED TO ARRAY 



3444 



ten 1x64 array of most significant cells 

i 

i 

I 

350 



CODING 



12/30 



3CID: <WO_970SS66A1 J_> 



WO 97/05566 



PCT/US96/11996 



C:\11_SYS\ABC\PATENTVENROLL\ECODE004 AF3 Fig. 12 CODE TRANSFORM 
Thursday, March 2. 1995 OUTPUT 
2:01 PM 



TRANSFORM 



^340 



CA\ riTT. ATE 
VECTORS <St 
RESIDUALS 



3502 

-J 



TM 



ONE-ON-ONE 350 



CODE OUTPUT 

- output: 1 . parameter vectors and 
corresponding standard deviations. 

• function: to code the transform output 
per parameter. 62 parameters 2 vectors per 
parameter with corresponding standard 
deviation. 

- input: standardized 1x64 arrays, known 
as parameters (for a ID Ao.I. one for a 2D 
Ao.I ten) 2 GENERAL EIGENSPACE 
SAMPLE. 



GEN EIGENSPACE 

62 BGENSPACES ONE PER 
PARAMETER 



PROJF.rT 

UN PARAMETER 
SPACE 



3504 



355 




3506 



, DEVIATION 
: CALCULATE 
: STANDARD 
DEVIATION 



WFIfiH 



3508 



ENCRYPT 



370 



VECTORS 



13/30 



OCIO: <WO 9705566A1J_> 



WO 97/05566 



PCT/US96/11996 



C\11_SYS\ABC\PATENT\ENROLL\EECRP004 AF3 fjg -|3 ENCRYPT 
Friday, March 3, 1995 
12:24 PM 



CODING 



350 



TM 



ONE-ON-ONE 



370 



ENCRYPT 
- output: I. encrypt key 2. verfication data 
: - function: determine useful parameter 
vectors and 

. - input: parameter vectors and 
■ corresponding standard deviations 



ENCRYPT RULE 



DFTFK MTNF 

USABLE PARAMETERS ■ 



3702 



3704 



370 



WRTTF 



[ENCRYPTION KEY A 
i VERIFICATION DATA j 



CARD 



3706 



70 



14/30 



9705566A1 I > 



WO 97/05566 



PCT/US96/11996 



CM 1_SYS\ABC\PATEhmVER!RVERIF004.AF3 
Wednesday, March 15, 1995 ~~ 
9:51 AM 22 



TM 



PRFPRnr.F.q<;iMn 

DIGITIZING SENSOR INPUT 



Fig. 14A VERIFICATION 

ONE-ON-ONE AP O 

VERIFICATION 



402 



SENSOR INPUT 

I SUCH AS AN IMAGE 
FROM A CAMERA 



WHOLE IMAGE 

STANDARDIZED NOISE 
FREE IMAGE 



305 



eye template 



AVERAGE EYE 

(DATA STORED 



-BINARIZE- 



ENHANCING CONTRAST 



binarized image centre 

. * 

: TARGFTING 

. LOCATING IRISES 



J) 



setup data: 
color thresholds 

310 



I 



CARD 

IV SETUP DATA 

2.). VERIFICATION DATA 

(PARAM. KEY A VALUE) 



setup data: 
likely face 
area * 



320 



y setup data 

targeting 

2 valid irisses with XY coordinates validation - 



setup pointer 



i SETUP CONTROL' 

r PROCESS CONTROL : 
; ARRAY. THIS SETS 

1 PROGRAM SWITCHES ; 

\ ACCORDING TO CLASS : 



-CALCULATE UD/CP-i 

m CALC. UNIT DISTANCE AND ! 
CENTRE PfONT 

valid UD&CP 

* 

: DFFfNF A n I 

AE£A of INTEREST 
. SEPARATION 



325 



setup data: 
selected 
330 A-oL 




410 

\ ' 
\ ! 
\ verification 

setup \ data" 
data: ! 
decision \ 
rules \ \ 



A.o.1. 
T 



-RESIZE A.o.t- 



335 



NORMALIZE A.O.L 



ID 8X64 / 2D 64X64 pixels 

i 



342_ 



EITHER ID FFT- 



TRANSFORMS ~ AA 



OR 2D DCT • 



OR PATREN 



T11 ■ T12 



T1N 



T21 y T22 - T2N 



Ti1 



ON j 

Ti2 \ / "tinN 



REGOCNTTK 



i \ i 
' 1 : 



^ x y 



■ i 
1 

transform output ID one 1x64 / 2D ten 1x64 arrays 



CONTINUE NEXT PAGE 



decrypt 
key 



15/30 



XIO: <WO 9705566A1J_> 



WO 97/05566 



PCT/US96/11996 



transform output ID one 1x64 / 2D ten 1x64 arrays 



Fig. 14B VERIFICATION 

- > 



»0 



EIGENSPACE 

(DATA STORE) 

ONE EIGENSPACE FOR 

EACH PARAMETER (=62) 



CODING 



CLASSIFYING OUTPUT 
Ao I TRANSFORMS 



decrypt 

keyi 



"parameters used 



\ 



verfication data i 



classified parameter values 



f COMPARE 

"ANALYSES OF EACH" 
TRANSFORMED 
PARAMETER VALUE &- 
CORRESPONDING 
.CARD VALUE 



J? 



DEC RYPT 

wrrH key 



RECONSTRUCT 
PARAMETERS USED 
AND THEER VALUES 



420 



values 



parameter cross correlation 



r DECISION 

RULES BASED 

IDENTIFICATION 

DECISION 




/ 

setup data: 
decision rules 



identification decision 



45( 



EXT. CONTROL 



/ 



16/30 



9705566A1_L> 



WO 97/05566 



PCT/US96/11996 



CM 1_SYSWBC\PATENT\VERIRVPREP0O4.AF3 
Fri, Mar 17, 1995 
9:57 AM 



Fig. 15 PREPROCESSING 



I SETUP CONTROL 



410 



22 



I 



SENSOR INPUT 

SUCH AS AN IMAGE 
FROM A CAMERA 



color / lumens 
threshold 



DIGITIZE 



4002 

-J 



TM 



ONE-ON-ONE 



PREPROCESSING 

- output: a standardized noise free 
digital image matrix 512 X 480 8 bit 
256 gray scale. 

- function: digitizing, filtering and 
enhancing the image from the camera 
to the enrollment process. 
Preprocessing is external to the 

: enrollment process. It is solely 
dependent on the input device and the 
frame grabber. All preprocessing takes 
place on the frame grabber. 

- input: 1 . analogue signal from the 
camera 2. setup data color / lumens 
threshold. 



FILTER 



4004 



402 



ENHANCE 



4006 



305 



COMPLETE IMAGE 

" COMPLETE NOISE FREE IMAGE 
MATRIX 512 X 480 8 BIT 256 
GRAYSCALE 



VERIFICATION i 



17/30 



OCID: <WO 9705566A1 J_> 



WO 97/05566 



PCT/US96/11996 



C:\1 1_SYS\ABC\PATENTVVERIRVSTUP004.AF3 
Friday, March 17, 1995 

70 



Fig. 16 SETUP CONTROL 



1:01 PM 



CARD 



setup pointer 



410 



RFAD 



4102 

* ) 

ARRAY y 



TM 



410 



ONE-ON-ONE 



SETUP CONTROL 

- output: setup data for the verification 
system (process control data) I . color / 
lumens thresholds 2. likely face area 3. 
targeting validation 4. selected A.o.L 5. 
decision rules 6. decrypt key 

- function: determine the program switch 
settings, based on the setup pointer. Pointer 
is the location of a cell in an array in 
program memory, containing the switch 
settings. Program reads setting and 
distributes the to the appropriate 
sub-functions of the verification process. 

- input: setup pointer 2 byte pointer (short 
interger) 



switch Bettings 



nr.STRiBinr 



4104 



color / lumens 
thresholds 



[ 



4p2 



PREPROCESSING ; 



identification decision rules 

i — 1 — 7° 

I DECISION 



decrypt key 
DECRYPT i 



likely face area 



^310 targeting validation 



BIN ARIZE 



TARGETING 



320 



selected A.ol. 



DEFINE A.o.1 ■■ 



18/30 



9705566A1I > 



WO 97/05566 



PCT/US96/11996 



C:\1 1_SYS\ABC\PATENT\VERIRVDCRP004.AF3 
Friday, March 17, 1995 
11:57 AM 



70 




Fig. 17 DECRYPT VERIFICATION 
DATA 

TM 

ONE-ON-ONE 420 

<->■ 

DECRYPT 

- output: parameter cross correlation 

- function: compare locally acquired data 
with card stored data for verification 

* input: 1 . locally acquired classified 
parameter values 2. the card stored 
parameter values (verification data) 



verification data 



DFPRVPT 



420 

j 



BASED ON DECRYPT 
KEY READ VALUES 
FROM CARD 



parameters used 



decrypt key 



f 



SETUP CONTROL 



410 
■J 



parameter values 



^0 



CODING 



430 



COMPARE 



19/30 



OCID: <WO 9705566A1 I > 



WO 97/05566 



PCT/US96/11996 



C:\1 1_SYS\ABC\PATENT\VERIRVCOMP004.AF3 
Friday, March 17, 1995 
11:57 AM 



^350 



\ 



CODING 



Fig. 18 COMPARE PARAMETERS 



TM 



ONE-ON-ONE 



430 



COMPARE 

- output: classified parameter values 

~ function: comparing the locally acquired 
parameter values to the card stored 
parameter values, yielding the difference. 

- input: classified parameter values 



classified parameter values 



rOMPARF ^ I DECRYPT 

^ values 



f 



parameter; cross correlation 



DECISION | 



4^0 



20/30 



) 9705566A1J_> 



WO 97/05566 



PCTYUS96/11996 



CM 1_SYS^BC\PATE^mVERIRVDECI004.AF3 
Friday, March 17, 1995 
11:57 AM 



3° 



f 



COMPARE 



Fig. 19 IDENTITY DECISION 



ONE-ON-ONE 



TM 



4jt0 



ID DECISION 

- output: parameter cross correlation 

. - function: compare locally acquired data 
with card stored data for verification 

- input: I. locally acquired classified 
parameter values 2. the card stored 
parameter values (verification data) 



parameter cross correlation 



DECISION 

rules based id 
decision of the 
parameters 



440 



setup data 



decision rules 



f 



SETUP CONTROL 



410 



identification decision 



i f 

20NTROr 



EXT CONTROL 



21/30 



)CID: <WO 9705566A1_I_ 



WO 97/05566 



PCT/US96/11996 




\ 



FIG 



22/30 



9705566A1J_> 



WO 97/05566 



PCT/US96/11996 




23/30 

OCID: <WO 9705566A1_I_> 



WO 97/05566 



PCT/US96/11996 




24/30 



XCID: <WO 9705568A1_L> 



WO 97/05566 



PCT/US96/11996 



8 

^ 

OX 



N 

T 

o 
ox 



0_ 



o 

^4 



\ 



QI 



3 

O 

fD V) 

O X 



N 
-I 

o 

CD CO 

QI 



it* 



O 
r><0 
OX 



-i 
O 
WW 

OX 



o 

3 

O 
OX 



8 

<M 
-J 

ox 



;*6 



Yl 



M 

r:z 
On 



K 



IL 

H( 0 



o 
f> 



Of 



*~- 

25/30 



DCID: <WO 970SS66A1_I_> 



WO 97/05566 



PCT/US96/11996 




26/30 



OCIO:<WO 9705566A1 I > 



WO 97/05566 



PCT/US96/11996 




OCID: <WO 9705566A1_I_> 



WO 97/05566 



PCT/US96/11996 




_9705566A1J_> 



WO 97/05566 



PCT/US96/11996 




9705566A1 I > 



WO 97/05566 



PCT/US96/11996 




9705566A1_L> 



INTERNATIONAL SEARCH REPORT 



International application No. 
PCT/US96/11996 



A. CLASSIFICATION OF SUBJECT MATTER 

IPC(6) :G06K 9/80 

US CL : 382/118, 382/190 
According to International Patent Classification (IPC) or to both national classification and IPC 



FIELDS SEARCHED 



Minimum documentation searched (classification system followed by classification symbols) 
U.S. : 382/118, 190, 115, 116, 117, 124, 190, 228 



Documentation searched other than minimum documentation to the extent that such documents are included in the fields searched 



Electronic data base consulted during the international search (name of data base and, where practicable, search terms used) 



DOCUMENTS CONSIDERED TO BE RELEVANT 



Category* 



Citation of document, with indication, where appropriate, of the relevant passages 



Relevant to claim No. 



X, P 



Journal of Cognitive Neuroscience, Volume 3, Number 1, 
1991, M. Turk and A. Pentland, "Eigenfaces for 
Recognition/ pages 71-86, especially page 76 ("Summary of 
Eigenface Recognition Procedure"), page 81 ("Database of 
Face Images") and figure 12. 

US, A, 5,497,430 (SADOVNIK ET AL) 05 March 1996, 
entire document, especially cols. 5-6, 13-14 and figure 2. 

US, A, 5,351,303 (WILLMORE) 27 September 1994, entire 
document. 

Carnahan Conference on Security Technology, 1992, F. 
Prokoski et al., "Identification of Individuals by Means of 
Facial Thermography," pages 120-125. 



1-6 



1-6 



1-6 



1-6 



I x[ Further documents are listed in the continuation of Box C. | | Sec patent family annex. 



' Special categoric* of cited documents: 

A" document defining the genermi •tele of the ait which U not considered 

to be part of particular relevance 

' E" earlier document published on or after the international filing date 

L" document which may throw doubts on priority claim(s) or which is 

cited to establish the publication dele of soother citation or other 
special reason (as specified) 

CT document referring to an oral disclosure, use, exhibition or other 

P" document published prior to the international filing date but later than 

the priority date claimed 



date or priority 



later document published after the international filing 
dale and not in conflict with the application but cited 
principle or theory underlying the invention 

document of particular relevance; the claimed invention cannot be 
considered novel or cannot be considered to involve an inventive step 
when the document is taken alone 



document of particular relevance; the 
considered to involve an inventive step 
combined with one or more other such doe 
being obvious to a person skilled in the art 

document member of the same patent family 



when the document 



Date of the actual completion of the international search 
18 AUGUST 1996 



Date of mailing of the international search report 




11 SEPfflB 



Name and mailing address of the ISA/US 
Commissioner of Patents and Trademarks 
Box PCT 

Washington, D.C. 20231 
Facsimile No. (703) 305-3230 



Aulhojfajcd officer / 



Y PRIKOCKIS 
Telephone No. (703) 30S-4791 



Form PCT/ISA/210 (second sheet)(July 1992)* 



DC1D: <WO 9705566A1_I_> 



INTERNATIONAL SEARCH REPORT 



International application No. 
PCT/US96/11996 



C (Continuation). DOCUMENTS CONSIDERED TO BE RELEVANT 



Category* 



Citation of document, with indication, where appropriate, of the relevant passages 



Relevant to claim No 



IEEE Transactions on Pattern Analysis and Machine Intelligence, 
Volume 12, No. 1, January 1990, M. Kirby and L. Sirovich, 
"Application of the Karhunen-Loeve Procedure for the 
Characterization of Human Faces/ pages 103-108, 



1-6 



Form PCT/ISA/210 (continuation of second sheet)(July 1992)* 



XDCID: <WO 9705566A1_I_> 



CORRECTED 
VERSION* 



WORLD INTELLECTUAL PROPERTY ORGANIZATION 
International Bureau 




PCT 

INTERNATIONAL APPLICATION PUBLISHED UNDER THE PATENT COOPERATION TREATY (PCT) 



(51) International Patent Classification 6 : 
G06K 9/80 



Al 



(11) International Publication Number: WO 97/05566 

(43) International Publication Date: 13 February 1997 ( 13.02.97) 



(21) International Application Number: PCT/US9671 1996 

(22) International Filing Date: 19 July 1996 (19.07.96) 



(30) Priority Data: 

08/507,528 
08/507,528 



26 July 1995 (26.07.95) US 
19 July 1996 (19.07.96) US 



(71) Applicant: IDENTIFICATION TECHNOLOGIES INTERNA- 

TIONAL, INC. [-/US]; Suite 703, 2655 LeJeune Road, 
Coral Gables, FL 33134 (US). 

(72) Inventors: HAN, Kedu; Apartment A801. 9125 S.W. 77th 

Avenue, Miami, FL 33156 (US). HERTZ. David, B.; 441 
Valencia Avenue, Coral Gables, FL 33134 (US). VAN 
GELDER, Lex; Apartment 212, 2785 Tiger Tale, Coconut 
Grove. FL 33133 (US). 

(74) Agent: GROSSMAN, Jon, D.; Dickstein Shapiro Morin 
& Oshinsky, L.L.P.. Suite 400, 2101 L Street, N.W., 
Washington, DC 20037-1526 (US). 



(81) Designated States: AL. AM, AT, AU. AZ. BB, BG, BR, BY, 
CA, CH, CN. CU, CZ, DE, DK. EE, ES, FI, GB, GE, HU. 
IL, IS. JP, KE, KG, KP, KR, KZ, LK, LR, LS, LT, LU. 
LV, MD, MG, MK, MN. MW, MX. NO, NZ, PL, PT, RO. 
RU, SD. SE, SG, SI, SK, TJ, TM. TR, TT, UA, UG, UZ, 
VN. ARIPO patent (KE, LS, MW, SD, SZ, UG). Eurasian 
patent (AM, AZ, BY, KG, KZ, MD, RU, TJ, TM), European 
patent (AT, BE, CH, DE. DK, ES. FI, FR, GB, GR, IE, IT, 
LU. MC, NL. PT, SE), OAPI patent (BF. BJ, CF. CG, CI, 
CM, GA, GN. ML. MR, NE, SN, TD, TG). 



Published 

With international search report. 



(54) Titfe: SYSTEM FOR OBJECT VERIFICATION AND IDENTIFICATION 



(57) Abstract 

The present system identities an object, 
such as a face, from unique signals and values 
emitted by the attributes of components on 
the object such as eyes, eyebrows, hairline, 
mouth, cheeks, ears, and chin (22). The object 
image is analyzed to determine the different 
components (325). These components are 
then transformed (340) through one of several 
transformations such as geometric, cosine, or 
Kohonen's, to yield ^ ^:ique description for 
each component. The data obtained is then 
transferred to either a portable memory, ID 
card or data base. A decision process of 
comparing the two results, from the object 
image, and from the data encoded in the ID 
card or data base is subsequently performed. 
In one example, this technology identifies 
human faces to verify the authenticity of an 
identity and then to grant or deny authority to 
carry out actions. 
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SYSTEM FOR OBJECT VERIFICATION AND IDENTIFICATION 

This application is a continuation-in-part of 
application Serial No. 08/507,528, filed September 26, 
1995, pending, the entire disclosure of which is 
5 incorporated herein by reference. 

FIELD OF INVENTION 
This invention relates generally to information 
storage and retrieval computer and. camera systems for 
obtaining and storing certain data relating to,, specific 
10 images and providing rapid access to that data for 
purposes of object targeting, enrollment, 
identification, verification, and classification. 

BACKGROUND OF THE INVENTION 

15 Many methods have been developed for 

commercial, military, and medical visual inspection 
applications. One particular area of interest for 
recent development efforts has been the area of image 
recognition. The basic idea behind image recognition 

20 xs to provide a system that captures data relating to 
an image area of an object or person and then compares 
that captured image data to information stored in a 
computer memory. 

However, many image recognition systems have 

25 drawbacks. First, they are characteristically slow, 

and subject to serious recognition and/or - ■ •- - 

identification errors. This is, in part, due to the 
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background of the art. The fact that prior methods 
have, for example, processed the entire human face, and 
have, used quantitative data obtained to effect the 
process of identification has resulted in very complex 
5 processor-bound systems. As a consequence, these 
systems are prone to error, and are cumbersome and 
costly. For example, the predominant facial 
recognition systems, which are Image Recognition, 
Pattern Recognition, and Retinal Recognition all rely 
10 on a central data storage, in the form of databases, 

which contain either a computerized copy of the entire 
facial image of the persons who are to be identified, 
or some abstracted form of the entire face or portions 
of the face to be reconnected later. The data for such 
15 systems is substantial for each face. Retrieval of 

data for these systems is time consuming, particularly 
as the number of enrolled persons increases. 

An example of a system exhibiting these 
drawbacks is Kodak's card-based facial security system. 
20 As understood, the Kodak system classifies fifty areas 
of the card holder's face, identifying each area with a 
2-byte code. That data is then stored on the stripe of 
a magnetic card. The card user must then have his/her 
face compared with the stored facial code in order for 
25 a match to be made. A drawback to this technique is 

that^ it requires the local computer to recognize 

multiple areas of the face, then classify each of these 
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areas, and then compare the instant classification to 
the code stored on the card. Hence, a significant 
processing burden is required at the recognition 
station. Moreover, the Kodak system is relatively 
5 inflexible in the sense that it is limited only to 

those things that have been classified. Thus, for the 
Kodak system to operate on other objects, a whole new 
classification scheme needs to the developed. The 
classification effort is a significant, labor-intensive 
10 process. 

Other known verification processes involve the 
comparison of large image structures stored in the 
system's database. The techniques used are various, 
but essentially work with the same data sets. The 

15 costs involved are high due to the complexity of the 
databases, the large size requirements for physical 
storage devices to contain these databases, and the 
necessity for fast computing speeds to process the 
information for real-time use. 

20 Object recognition against a known data set is 

a relatively easy art for humans to master, but 
presents difficult problems for computers. For a 
technique that permits a computer to carry out a 
successful recognition process, the process must yield 

25 quantifiable numerical descriptors. This process of 
checking against the known data set would/" of course, 
also involve a quantifiable registration and enrollment 
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of the object- The data set generated during the 
enrollment process is used for checking against the 
generated data from the recognition and verification 
systems. Yet, the numerical data produced by the 
5 digitized object images, such as human facial images, 
can be large. Once about a thousand complete images 
are stored, the retrieval and comparison of the data 
sets can become uneconomical and time-consuming. 

To avoid these drawbacks, certain systems, such 

10 as exemplified by U.S. Patent No. 5,161,204 or U.S. 
Patent No. 5,164,992, have resorted to complex 
classification schemes in order to extract key features 
and to perform discrete calculations for identification 
and verification. However, such schemes increase the 

15 processor overhead that is necessary to extract 
components from the image field. They do not 
necessarily reduce the storage requirements necessary 
to retain a database of captured faces. Thus, there is 
a need in the art for a verification and identification 

20 system that allows for high reliability, high accuracy 
and low cost, but at the same time is efficient. 

SUMMARY OF THE INVENTION 

In view of the foregoing, there exists a need 

in the art for a system that does not attempt to 
25 develop identification and/or verification using entire 
object feature comparisons.,, but instead utilizes 
selected data sets from components of the object for 
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comparing the stored data sets with input images. Such 
combinations of data sets are to be used to uniquely 
identify an object. The independence of the attribute 
data sets leads to a normative analysis that can 
5 compare the data sets generated from input with data 
sets from known objects or persons. 

It is another object of this invention to 
provide a comparison process that involves polling 
various attribute data against a known data set. 

10 It is yet a further object of the invention to 

provide a system and method for obtaining, digitizing, 
and recording selected data sets of separate and 
independent component element image signals, sensed by 
photographic or electronic means, from a person or 

15 object, for subsequent comparison with similarly 

obtained component elements image signals of a person 
or object whereby the degree of similarity between the 
original and the subsequent data sets may be 
determined. 

20 -1C is yet a further object of the invention to 

provide a method whereby data sets of sensed images may 
be recorded on portable media, such as magnetic stripe 
cards, magnetic discs, printed bar codes, semi-conductor 
devices such as smart cards, or in data bases. 

25 It is an additional object of the invention to 

- provide a method of selecting separate and independent 
component elements of persons or objects whereby the 

SUBSTITUTE SHEET (RULE 26) 



DCID: <WO 9705566A1 JA> 



WO 97/05566 6 PCT7US96/1 1996 

selected components provide image signals for recording 
quantitative structures or arrays of numbers that permit 
statistical identification of the individual person or 
object from among data sets previously obtained from the 
5 same person or similar objects. 

It is yet a further object of the invention to 
provide a method whereby the image signal data sets of 
persons or objects are measured so as to permit the 
determination of a quantitative degree to which the 
10 objects or persons are similar to those originally 
measured . 

It is an additional object of the invention to 
provide a method whereby it may be determined whether 
component image signals obtained from a person or object/ 

15 at any time subsequent to the recording of a set of such 
image signals from the person or object, differ, or do 
not differ, from the recorded component images by any, or 
by more than or less than, preselected quantitative 
values, permitting the system to indicate that the 

20 signals may be quantitatively, or qualitatively, 

verified, or not verified, as being from the recorded 
signals of the person or object. 

It is a further object of this invention to 
provide a camera and infra-red LED array which is 

25 designed to optimize the lighting and imaging of a 

particular . area of the object. 
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It is another object of the invention to provide 
a system whereby quantitative measures may be selected 
for inclusion in the digital data system for allowing the 
determination of chosen levels of difference between 
5 subsequently obtained component image signals and 

verification of such signals as being, or not being, from 
the recorded signals of a specific person or object. 
It is another object to provide an 
identification and/ or image verification system that 
10 uses a method of transforming various independent 

attributes of an object, such as, for example, a nose 
on a human face, into a data set for a component. For 
the whole object the various components are independent 
of each other. The data sets are generated during 
15 registration or enrollment. These data sets are 

recorded on a card capable of carrying data, such as a 
magnetic strip or a 2d-barcode, that is issued to a 
holder. In effect, the advantage of the system is that 
instead of maintaining a central database, the 
20 identification data is now decentralized and held on 

the small cards. Also, in the process of transforming 
the object component attributes into data sets, a 
dramatic reduction in the data that is required to be 
identified is achieved. The reduction of the data, 
25 required to uniquely identify complex objects such as 
human faces, also achieves a faster response in the 
identification process. 
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It is a further object of the invention to 
obtain the data sets through various transformations 
that are specific to the object component attribute 
chosen. The comparison process involves the 
5 verification of data present on the ID card, with the 
data sets generated from the video or other image of 
the object that has been registered through an input 
device such as an electronic camera. The comparison 
process utilizes a neural network that has been trained 
10 so as to recognize or identify a particular data set 

(such as human facial image component attributes) . The 
training of the neural network is based on a process of 
polling the various attributes that are obtained at the 
identification station by the computer, against the 
15 component attribute data sets that are present on the 
ID card. The polling assumes that certain distinctive 
features, if in agreement with the data sets on the ID 
card, can override other less distinctive attributes. 
However, as the security needs of the application 
20 increases, the polling process increases the required 
precision of the comparisons. In one embodiment, a 
program is provided for reducing the characteristics of 
an object image, for example, a human face, to a set of 
characteristic numbers; later recalling that set of 
25 characteristics for comparison with an input from an 
... external source. Keys to this set of numbers are 
encoded as indices and are stored on local source 
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objects, such as 3-3/8" x 2" computer input cards. The 
indices, having been electronically posted to a central 
computer program, point to a second set of data 
retrieved from the computer program. A comparison of 
5 that set then occurs with a second set of similar 

stored data retrieved from a local source, such as the 
card. 

These and other advantages and features of the 
invention, the various embodiments, and other aspects 
10 of the invention, should become more apparent from the 
following description, drawings and claims. 

BRIEF DESCRIPTION OF THE DRAWINGS 

FIG. 1 is a block diagram of the enrollment 

station forming the present invention; 
15 FIG. 2 is a block diagram of the verification 

station forming the present invention; 

FIGS. 3A and 3B are flow-chart diagrams showing 
the functions of the enrollment process; 

FIG. 4 is a flow chart diagram illustrating the 
20 one-on-one picprocessing steps of the enrollment 
process shown in FIGS. 3A-3B; 

FIG. 5 is a flow chart diagram of the 
binarization routine of the enrollment process shown in 
Figs. 3A-3B; 

25 FIG. 6 is a flow chart diagram of a first 

embodiment of the targeting process for the enrollment 
process shown in Figs. 3A-3B; 
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FIG. 7 is a flow chart diagram of the UD and CP 
coordinate estimation functions of the enrollment 
process shown in FIGS. 3A-3B; 

FIG 8 is a flow chart diagram illustrating the 
5 area of interest defining function of the enrollment 
process shown in FIGS. 3A-3B; 

FIG, 9 is a flow chart diagram illustrating the 
normalization procedure of the enrollment process shown 
in Figs. 3A-3B; 

10 FIG. 10 is a flow chart diagram showing the 

transform step of the enrollment process shown in Figs. 
3A-3B; 

FIG. 11 is a flow chart of the second 
transformation process for the enrollment process of 
15 Figs. 3A-3B; 

FIG. 12 is a flow chart of the output coding 
function for the enrollment process of Figs. 3A-3B; 

FIG. 13 is a flow chart illustrating the 
encrypt function for determining useful parameter 
20 vectors for the enrollment process of Figs. 3A-3B;' 

FIGS. 14A-14B are flow charts of the process 
for image verification of the present invention; 

FIG. 15 is a flow chart of the image 
verification pre-processing function; 
25 FIG. 16 is a flow chart of the image 
verification setup control function; -. • 
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FIG. 17 is a flow chart of the image 
verification data decryption function; 

FIG. 18 is a flow chart of the image 
verification parameter value comparison function; and 
5 FIG. 19 is a flow chart of the image 

verification identity decision function; 

FIG. 20 is a diagram showing the dimensional 
breakdown of the face; 

FIG. 21 is a top view of the array of infra-red 
10 light emitting diodes used to light a mini-camera 
apparatus; 

FIG. 22 is a perspective transparent diagram of 
the mini-camera and infra-red lighting device and 
components thereof of the invention; 
15 FIG. 23 is a circuit schematic diagram for the 

light array of FIG. 21; 

FIG. 24 is a circuit schematic diagram for the 
mini-camera of FIG. 21; 

FIGS. 25 (a) -(c) are respectively front, side 
20 and perspective tfiews of a first embodiment of a 

housing for the mini-camera arrangement shown in FIG. 
22; 

FIGS. 26 (a) -(c) are respectively front, side 
and perspective views of a second embodiment of a 
25 housing for the mini-camera arrangement shown in FIG. 
22; 
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FIGS. 27 (a) -(c) are respectively front, side 
and perspective views of a third embodiment of the 
housing for the minicamera arrangement of FIG. 22; and 
FIG. 28 is a flow chart illustrating a second 
5 embodiment of the targetting process for the present 
invention . 

DETAILED DESCRIPTION OF THE INVENTION 

Referring now to the drawings wherein like 

reference numbers refer to like parts, the first 

10 embodiment of present system is composed of two 

processes and two hardware systems. The first process 
of this first embodiment is the enrollment process and 
is shown in Figs. 3A-13. The purpose of the enrollment 
process is to code the image of the person or object 

15 and to reduce that image to a portable form, and 

format, such as a card with a magnetic strip, or in a 
database. The second process of the first embodiment 
is shown in Figs. 14A-15. This process is the 
verification process. The verification process 

20 performs the tasks of taking a picture or image of the 
person or object, and comparing the captured image to 
the coded image obtained from the enrollment process. 
If the image of the person or object obtained during 
verification and the coded information obtained from 

25 the enrollment process match, identity is verified by 
. the verification process. The enrollment process and - 
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the verification process have elements in common, which 
will be described further below. 

The two hardware systems used in the present 
invention are the enrollment station, shown in Fig. l, 
5 and the verification station, shown in Fig. 2. 

Fig. 1 is a block diagram of the enrollment 
station 100. The object 10 represents the object that 
will be coded in the enrollment process for later 
verification during the verification process. In a 

10 preferred embodiment, the object 10 is a face of a 
person. However, any object with distinguishing 
features can be used with this invention. For example, 
the object under consideration may constitute a machine 
part under inspection, or a house, or a car key, or an 

15 automobile, or a hand. As another example, in 

warehouse operations, it is essential that objects are 
correctly labeled, stored, and shipped. The label on 
the object or its container carries data related to 
features of the object. These data permit an automatic 

20 verification that rhe object is correctly labeled, and 
therefore will be properly stored, packaged, and 
shipped. Object types can vary, as long as an 
identifiable characteristic can be extracted, and is 
stored in the enrollment process. In order to enroll 

25 the person or object, one or more video cameras 20 or 
other cameras as set forth in this application and — 
equipped with lighting devices 30, are used to record 
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an image of the object 10. The video camera or cameras 
20 and lighting device 30 are ordinary devices that are 
readily available. In the first embodiment, Panasonic 
camera, CCD Model No. GP-MF602, or similar devices are 
5 used with either flash or continuous light sources. 
The lighting devices can in this first embodiment 
comprise a ring lamp such as the MICROFLASH 5000, 
manufactured by VIVITAR Corp., located at Chatworth, 
California, or a standard photography lighting fixture. 

10 Other camera devices and lighting devices, however, can 
be substituted. For example, a flash device can be 
employed with a Panasonic camera. An example of a 
flash unit is the SUNPAK Softlite 1400M, manufactured 
by TOCAD Company, Tokyo, Japan. Alternatively, a 

15 continuous incandescent light source can be employed. 

This type of lighting device is particularly useful in 
conjunction with object identification/ verification for 
quality control applications. Finally, as will be 
described in Figs. 21-27 of this application, an LED 

20 lighting device can be employed in conjunction with a 
mini infra-red camera. 

The output of the video camera 20 is connected 
via a port to computer 40. The computer 4 0 can be a 
personal computer that includes a digitizer card 42 in 

25 an expansion port. The computer also includes other 

standard components, such as a CPU 44, a random access 

memory 4 6, and a permanent storage, in the form of a 
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magnetic disk storage 48, a monitor 50, and a keyboard 
52 . 

In the preferred embodiment, a personal 
computer is used having an Intel® Pentium® 
5 microprocessor designed to have a minimum processor 
clock speed of 90 MHz. The computer has in this 
example 32 MBytes of Random Access memory and at least 
1 Gigabyte of static storage. However, any combination 
of clock speed and memory size types can be used in 

10 this system. A conventional hard-drive can be used, 
although other static storage units (e.g., writable 
optical drives, EPROM) are also acceptable. A 
Microsoft Windows® operating system is used. It should 
be noted that the present invention is designed so that 

15 any computer having adequate processor clock speed 

(i.e. preferably a clock speed of at least 60 MHz) , and 
a sufficient RAM memory size of (i.e., 16 megabytes of 
RAM) can be employed. 

The digitizer 42 used is a frame grabber card 

20 for transforming camera signals from analog to digital 
form. This card is inserted in the computer 40. In 
the preferred embodiment, the card used is a CX100 
Imagination Board, manufactured by Image Nation 
Corporation, located in Beaverton, Oregon. However, 

25 any digitizer device for video input including direct 
input from digital video cameras v can be used with the 
invention . 
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The output device 60 receives the data from the 
computer 40 which is a coded representation of the 
object 10. Device 60 transforms the coded data into an 
appropriate signal and code for placement on a central 
5 storage 72 or portable memory device. The output 

device 60 receives the data from the computer 4 0 which 
is a coded representation of the object 10. Device 60 
transforms the coded data into an appropriate signal 
and code for placement on a central storage 72 or 

10 portable memory device 70. The output device 60 in the 
preferred embodiment is a card reader/writer, or a 
printer. Examples of output devices include magnetic 
or optical reader/writers or smart card reader/writers 
or barcode printers, etc. 

15 In the preferred embodiment, that memory device 

70 is a magnetic stripe card. The output device 60 and 
card 70 are well known in the art. However, other 
portable memory devices such as optical cards, S-RAM 
storage devices, or carriers containing EPROM or UVPROM 

°0 memories, can also be used. In addition, known 

barcoding schemes can be employed to bar-code the 
information, where appropriate. 

Fig. 2 is a block diagram of the verification 
station hardware 200. The data of an enrolled person 

25 or object 210 appropriately lighted by lighting 30 will 

... . be compared to the coded representation of the object 

on the portable memory 70. The object 210 represents 
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the same object as the enrolled object 10 in Fig. 1. 
The purpose of the verification station 200 is to 
output to an appropriate security system, a signal via 
external control 230. For example, the external 
5 control 230 can be an electronic access device, such as 
an electronic door lock, or a motor-driven gate, or any 
other mechanism. The components in the verification 
station 200 are the same as the components of the 
enrollment station 100 with the exception of the input 

10 device 220 and the external control unit 230. 

The card 70 is inserted in the input device 
220, such as a magnetic or optical card or a barcode 
reader, while the object 210 has its image recorded and 
processed by the computer 40. Image recordation and 

15 processing is done in an identical way as in the 

enrollment station discussed above A program in the 
computer 40 for the verification station compares the 
image data of the object 210 with the coded image data 
on the card 70. If there is a satisfactory match, the 
c 20 verification signal 230 is outputted indicating a 
match, or a failure to match. 

Figs. 3A and 3B are overview flow chart 
diagrams showing the process used by the enrollment 
station 100 in order to encode enrollment data onto the 

25 portable storage medium 70 or central database 72. 

Each of "the steps of Figs. 3A-3B are detailed in the 
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flow charts of Figs. 4-13. Each overview step will be 
elaborated on below by reference to the later figures. 

Cameras 20 provide input for the computer 40 
which executes the preprocessing function 302. This 
5 preprocessing function 302 is described in detail in 
Fig. 4. 

Specifically, in Fig. 4, the preprocessing 
function 302 is performed by the computer 40 in 
combination with a frame grabber digitizer 42. The 
10 frame grabber digitizer 42 first transforms the analog 
signal at step 3002, then filters the digitized data at 
step 3004, and enhances the image at step 3006. The 
filtering step 3004 filters out image noise using 
conventionally known techniques. The enhancement step 
15 3006 enhances image contrast according to the lighting 
conditions, using known techniques. The output of 
these functions is the complete image 305 which is 
composed of a standardized noise-free, digital image 
matrix of 512 x 480 pixels. The complete image 305 is 
20 used as an input to rious other subroutines in the 
enrollment process which will be described below. 

Returning to Fig. 3A, the next step in the 
enrollment process is the binarization process 310. 
This process is illustrated in detail in Fig. 5. The 
25 function of the binarization process 310 is to convert 
8-bit 256 gray scale pixels of the input matrix into a 
two-level color output. As shown in Fig. 5, step 310 
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receives as the input matrix the complete image 305. 
At step 3102, a center image is taken from the complete 
image. The coordinates of the upper left hand corner 
of the center matrix is defined as coordinates 128 x 
5 120 and the coordinates of the bottom right hand corner 
of the center matrix is defined as coordinates 384 x 
360. This central image is then binarized. This 
process results in an image where each pixel is either 
black or white, depending on whether a pixel exceeds or 
10 falls below a preset threshold. 

As noted, the coordinates are chosen for the 
preferred embodiment to focus on the center of the 
image ♦ However, if the object does not have 
distinguishing features in its central area, different 
15 coordinates can be used. This output image is then 

made available to the targeting procedure 320 shown in 
Fig. 3A of the enrollment process. 

An alternative embodiment for the targeting 
process is also shown in Fig. 28. The targeting 
" 20 procedure 320 is sho v wh in more detail in Fig. 6. The 
purpose of the targeting procedure is to find a 
distinguishing feature in the object in order to detect 
the presence of the object and determine the location 
of the object in the image matrix. In the preferred 
25 embodiment, the distinguishing features looked at are 
the two irises of the eyes. As shown in Fig. 6, the " 
input to the targeting function 320 is the binarized 
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image 3104 which is then processed by the labeling 
function 3202. In the preferred embodiment the 
labeling function 3202 locates and labels all areas 
that exhibit similar characteristics as irises. In 
5 other words, the threshold set for the binarization 
process 3102 is set to filter out gray scale levels 
that are not relevant. Thus, the gray scale color 
typically associated with irises can be used as the 
indicator for the threshold. The output of the 
10 labeling process 3202 comprises the object labels 3204. 



Next, the coordinate calculation process 3206 
is activated. In this step, each labeled object 
produced at step 3204 has the XY coordinates calculated 

15 for placement in the complete image matrix 305. This 
provides a geometric center of each object that was 
labeled in the previous step. 

Thus, in the first embodiment, in step 3206 the 
irises are located and are distinguished by their 

20 contrast with the surrounding area. In addition, other 
contrasting areas may also be labeled. These 
contrasting areas, are for example in this exemplified 
application, nostrils or lips. 

The next step in the targeting process 320 of 

25 Fig. 6 is the coordinate validation step 3208. Step 
3208 involves looking at the XY coordinates of a pair 
of objects and then determining whether their absolute 
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and relative locations are valid. The validation step 
3208 assumes that labeled objects, such as the irises, 
are appropriately positioned on a face. For example, 
the eyes cannot be on top of each other and must fall 
5 within acceptable distances from each other. 

Therefore, the validate coordinate step 3208 function 
determines those pairs of labeled objects that can 
possibly be irises. 

Specifically, the calculations for iris 

10 targeting consists of comparing the XY coordinates for 
each iris to determine if they are within a preset 
distance apart and on approximately the same horizontal 
axis. In addition, the difference in the X coordinates 
are measured and compared to a prestored value to make 

15 sure that the irises are located at certain specific 
locations. In the preferred embodiment, the 
coordinates Yl and Y2 represent the horizontal 
coordinates, and XI and X2 represent the vertical 
coordinates. Thus, Y2 and X2 in the preferred 

20 embodiment represent the left iris coordinate, and Yl 
and XI, the right iris. 

A first calculation determines if Y2 is greater 
than Yl . In the second calculation, the result of Y2 
minus Yl should be greater than a value of 40 pixels. 

25 The third calculation determines the absolute value of 
XI minus X2. In the preferred embodiment, that value 
should be less than 16 pixels. If all three equations 
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are met, then at step 3208 the object's pair of irises 
is confirmed, and processing passes to step 3216. 

However, if no valid targets are found (failure 
to pass at least one of the above three equations) an 
5 output message is sent at step 3212 to monitor 50 (Fig. 
1) stating that the process has been unable to target 
the eyes. A new image is acquired again and 
reprocessed, beginning at step 302. 

The next step 3216 is to validate the object. 
10 This step compares the spots with the eye template to 

determine whether a cross correlation coefficient fits. 
If so, it confirms that the system successfully 
targeted the eyes. 

Any one input to the validate object step 3216 
15 is determined at step 315. This input is an average 
eye template value, which is an average of the iris 
position on the face across a wide population. The 
other input, determined at step 305, which was 
discussed previously, is the complete image. As 
20 noted, the complete image ir - reference standardized 

noise-free image matrix of 512 x 480 pixels, 8-bit gray 
scale. The validate object step 3216 performs a gray 
scale correlation using the complete image 305 and the 
average eye template 315 and the valid object XY 
25 coordinates. This complete image is compared to the 
average eye template at the. valid XY coordinates. If 
the maximum correlation is above a preset threshold, 
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the object is identified as an eye. The correlation 
coefficient of two areas {A^} and ib^) is calculated 
as : 



where A tj and b t1 are pixels of the two areas. 
The threshold of correlation is 0.9. 
The outputs of this comparison are two "valid" 
iris values with the associated XY coordinates in the 
10 complete image matrix 305. The outputted values are 
provided at 3218. 

Returning to Fig. 3A, following the targeting 
process 320, the system then retrieves the calculated 
unit distance/center point by initiating the process 
15 set forth at step 325. A detailed flow chart of this 
process is shown in Fig. 7. 

Referring to Fig. 7, the calculate unit 
distance and center point routine 325 establishes a 
unit distance for the center point in the image based 
2o on the coordinates of the iris provided from the 

targeting step 320. The unit distance (UD) equals 
UX1-X2) exp 2 + (Y1-Y2) exp 2) exp The center 

point (CD) equals, for the X coordinate, CX = (XI + 
X2)/2 and for the Y coordinated CY = (XI + Yl)/2. 
25 The next step of the enrollment process shown 

in Fig. 3A, is to define an. area of interest at step 
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330- The area of interest procedure 330 is shown in 
detail in the flow chart diagram of Figure 8. 

Referring to Fig. 8, the function of step 3301 
is to define the areas of interest on the object in 
5 relation to the unit distance (UD) and center point 
values (CX and CY) . The areas of interest are 
predetermined depending on the object to be identified. 
In the preferred embodiment, eight areas of interest 
have been selected. These areas of interest are a 
10 one-dimensional horizontal bar on the forehead, a one 

dimensional vertical bar over the center of the face, a 
two-dimensional right and left eye section, a 
two-dimensional right and left eyebrow section, and a 
two-dimensional right and left cheek section. 
15 Essentially, the areas of interest for a face in the 
preferred embodiment are dissected into two 
one-dimensional areas and six two-dimensional areas of 
interest (see Fig. 20). 

Returning to Fig. 3A, once the areas of 
20 interest are determined, the r~**.t step is to normalize 
the areas of interest at step 335. Step 335, shown in 
detail in Figure 9, resizes the area of interest to a 
standard pixel size. In the preferred embodiment, the 
standard pixel size for the one-dimensional pixel area 
25 of interest is 8 x 64 pixels. For the two-dimensional 

areas of interest, the standard pixel size is 64 x.64 

pixels. The purpose of this normalization procedure 
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step is to standardize the input to the transform 
procedures . 

Step 340 shown in Fig. 3A then performs several 
transforms, each of which is applicable to particular 
5 areas of interest. One of these transform processes is 
step 342. Specifically, step 342 applies transforms to 
the one- dimensional pixel arrays (representing the 
eight areas of interest) outputted from the resized 
areas of interest step. 

10 As seen in Fig. 10, eight fast-fourier 

transforms { FFTs ) , as known in the art, are performed 
at step 3420 on the one-dimensional pixel arrays. The 
results of these transforms are averaged at step 3422 
into a 1 X 64 vector array representing the spatial 

15 distribution of that area of interest. Processing then 
returns to Fig. 3A. 

Another transform is performed at step 344 
specifically applied to the dimensional areas of 
interest. The transform used is a discrete cosine 

20 transform (DCT) , as known in the art (see Fig. 11). 

Referring to Fig. 11 , each 64 x 64 pixel array 
is divided into 64 separate pixel arrays of 8 x 8 
pixels at step 3440. Then each 8x8 pixel array is 
compressed using the DCT at step 3442. The output of 

25 the DCT for each 8x8 pixel array is a transformed 

array with the most significant cell in the upper left 
hand corner. Using all the 8x8 transformed arrays, 
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ten 1 x 64 vectors arrays of the most significant cells 
are then created at step 3444. Other techniques can be 
employed, such as edge detection, Kohonen ' s and/or 
geometrical analysis. Step 346 in Fig. 3A, depicts 
5 these other alternative transforms which can be used to 
compress and analyze identified areas of interest. 

For example, the most significant cell of each 
of the 64 transformed arrays comprises the first 1 x 64 
vector array, the second most significant cells 

10 comprises the second 1 x 64 array, and so on. The 

result is that each 64 x 64 pixel area of interest is 
transformed into ten 1 x 64 vector arrays of the most 
significant transformed cells. These arrays are then 
sent to the coding routine 350. 

15 As an alternative to the above, each layer can 

be binarized, so that if each cell's coefficient is 
greater than zero, then the value for that cell is 
equal to one. If that cell's value is less than zero, 
then it's binarized value is equal to zero. As a 

20 consequence, relatively few bytes for multiple layers 
are necessary. For example, if each layer is 8x16 
bytes, then the binarization will create and an 8x16 
bit layer. For a 6-layer image, for example, 96 8-bit 
bytes (6x16) will be created for the captured image. 

25 Thus, under this alternative binarization step a very 
small amount of memory, is necessary for the UD/CP 
values . 
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Referring now to Fig. 3B, the coding routine 
350 is shown. Fig. 12 sets forth routine 350 in more 
detail. Referring to Fig. 12, input to the coding 
routine array are the sixty-two, 1 x 64 vector arrays 
5 produced by the transform routine at step 34 0 (Fig. 

3A) . For each one-dimensional area of interest, one 1 
x 64 vector array is inputted. For each 
two-dimensional area of interest, ten 1 x 64 vector 
arrays are inputted. Therefore, in the preferred 
10 embodiment, 62 1 x 64 vector arrays are inputted to 
routine 350. 

The other input 355 is the eigenspace. The use 
of eigenspaces is well-known in the art as a method for 
determining the characteristics of an individual 

15 observation to a sample of the general population. See, 
for example, Kirby, M and Sirovich, L. Application for 
Karhunen-Loeve Procedure for the Characterization of 
Human Faces , IEEE Trans. Patt. Anal. Machine Intell., 
Vol. 12, pp. 103-108, 1990; Turk, M. and Pentland, A., 

2 0 Eigenfaces for Recognition Journal and Cognitive 

Neurosciences , Vol. 3, pp. 71-86, 1991; Gonzalez, R.C. 
and Woods, R.E., Digital Image Processing , 
Addison-Wesley, 1992 . 

The first coding step 3502 calculates residuals 

25 of the vectors. The residuals are the differences 
between the sixty-two vectors and the mean" vectors 
estimated for a general population. Next, these 
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residuals are projected into their sixty-two separate 
eigenspaces, permitting one per parameter. The result 
of this process provides the two most significant 
coordinates, per parameter, in their respective 
5 eigenspaces. In total, 124 coordinates are calculated. 
Process step 3504 is repeated several times to insure a 
statistically appropriate sampling of the enrollment 
images calculates the mean and standard deviation of 
the 124 parameters coordinates generated at step 3504. 

10 Step 3508 then evaluates the coordinates with the 

smallest standard deviation and highest coefficient 
with the average of the population. Based on those 
criteria, the coordinates and their respective weights 
are then passed to the encryption process 370. 

15 The encryption routine 370 of Fig. 3B is shown 

in detail in the flow chart of Fig. 13. Such a routine 
is well known in the art. For example, the encryption 
algorithm shown at step 3702 determines usable 
parameters according to encryption criteria 3704 which 

20 are related to the mean and the standard deviation of 
the parameter coordinates. The result is the 
encryption key and verification data which are written 
at step 3706 onto the portable storage 70. However, a 
code or any other well known technique in the art of 

25 recording information can be used. Therefore, the card 
70 contains the coded information pertaining to the 
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object that was enrolled. The enrollment process is 
now complete. 

Fig. 14A & Fig. 14B show an overview of the 
verification process using the verification station 
5 hardware shown in Fig. 2. Most of the procedures in 

the verification process are similar to the procedures 
previously discussed regarding the enrollment process. 
Thus, a detailed explanation is reserved for those 
processes that differ. A detailed description of the 

10 verification steps are set forth in Figs. 15-19. 

A prerequisite to the verification process 400 
is for the enrollment process to be repeated, up to a 
certain point. Specifically, in the preferred 
embodiment, the person that needs to be verified would 

15 go through step 302 (preprocessing) through step 350. 
The output of step 350 in the verification process 
provides parameter values corresponding to the images 
of the person or object to be verified. At the same 
time, card 70, which contains the data from the 

20 enrollment process, is inserted into a reader- device 

which then decrypts the magnetic data, yielding process 
control data (Fig. 16 step 410) and parameter values 
that correspond to each area of interest (Fig. 17, 
process 420) . The process control data instructs the 

25 machine on how to accomplish verification. The 
" " parameters values determine what is to be verified. The 
parameter values are compared to the output of the 
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coding step 350 in the verification process (Fig. 18 
process step 430) . Statistical analyses, as is 
generally known in the art, is used to accomplish 
verification. Other methods, however, such as 
5 ruled-based identification decision processes or fuzzy 
logic systems, may be used in addition to the 
straight-forward statistical analyses. Furthermore, a 
degree of correlation between the two values can be 
varied depending on the degree of sophistication of the 
10 verification technique desired. If at the decision 

making step (Fig. 19 process 440) the parameters of the 
card match the parameters of the photograph image, 
verification has been achieved and an output is made to 
the external control unit process 450. 
15 One verification methodology for example can 

rely on increasing the Hamming distance between the 
enrolled image and the image to be verified. In this 
instance, the image vector stored in the card, or other 
non-volatile storage media is lined up, bit-by-bit to 
the generated image vector. The bits for each vector 
are compared. For different bits, a value of "1" is 
generated, for identical bits, a "0". A Hamming 
distance is then generated, as follows: 

where n = total number of bits in the vector 



20 



25 



where N = total length of vecot 
(normalized) Hamming Distance (HD) = n 



N 
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The HD value can be used as a threshold value, 
from which system sensitivity can be varied. If the 
values are set at around, for example .23, other 
sensitivities for retest, for example, or for reject 
5 can also be set. If for example, accept is .23 (HD) , 
retest is .24 - .74 and reject is .75 or greater, then 
it is possible that over time the retests will migrate 
to either direction (i.e., accept, reject). 

Figs. 21-27 respectively illustrate a 

10 micro-camera assembly and LED lighting apparatus which 
provide numerous operational advantages both to the 
various embodiment of the invention, as well as to any 
other known image enrollment/recognition systems. 

In particular, Fig. 21 illustrates a front-view 

15 of an array of light emitting diodes ( "LEDs " ) 2102 
located along the same plane of a plate 2104. The 
arrangement of the LED's has a specific size, and 
intensity, to optimize lighting intensity of the target 
and capture of the image. In this diagram, a 

20 configuration is shown for maximizing iris capture. 
The LED's are designed to light the iris at a lower 
visible infra-red spectrum, rather than at the "heat 
detecting" spectrum. The spectrum tends to have a wave 
length of approximately 880nm, although other low 

25 visible spectra are considered applicable. Infra-red 

spectrum light has been found to be optimal, since ~ — 
flash lighting is distracting, if not painful. 
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Moreover, infra-red, at low levels, standardizes facial 
fill so that all features of the face are equally 
exposed. A further advantage to low level infra-red is 
that, when combined with an appropriate filter, ambient 
5 light is cut out altogether. Consequently, image 

wash-out is avoided. Finally, the higher spectrum heat 
detecting level. has been found to be less accurate in 
measuring biometric characteristics . 

As shown in Fig. 21, an array of nine LED 1 s 

10 2106 are arranged in a square that is angled a 45° 
relative to the horizontal axis 2108 of plate 2104. 

Fig. 22 is a transparent perspective view of 
the microcamera device 2200 incorporating the 
af oredescribed LED array 2100. The device includes 

15 four basic elements: a micro-camera lens 2202, a 

microcamera circuit board 2204, the af oredescribed IR 
LED array 2100 and the LED circuit board 2206. These 
four elements are contained in a housing 2208. 

The housing 2208 is designed so that the lens 

20 and LED array are heJd in a potting material in order 

that the microcamera unit may be contained, and sealed. 
As a consequence, the microcamera can be used in 
underwater applications without substantial water 
leakage to circuit boards 2204 and 2205. The potting 

25 has a sufficient circumferential clearance around the 

_. ._ lens element 2202 in order to allow ..the -lens to freely 

rotate . 
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The top surface of the housing 2210 contains a 
recess 2210 the top surface of which is co-planar with 
the top surfaces of the lens 2202 and LEDs 2102. 
Further, a pair of flanges are arranged parallel to 
5 each other and the longitudinal axis of housing 2208 so 
that a flat filter element (not shown) that is sized to 
fit between the flanges can slide across the top 
surface 2210 and be held in place by the flanges. The 
filter comprises a sheet of mirrored glass or plastic 

10 that is near infra-red. The filter is thus able to cut 
off the visible light spectrum. 

The mini camera housing includes a 
communications port 2220 which provides the image 
output to an external frame grabber. The port 220 also 

15 connects to an external power supply for the 

mini-camera. The port 220 may use any optimal 
wiring-confirmation which in this embodiment a 9 pin 
DIN that can connect to the I/O port of a PC. 

In a second embodiment, the camera device 22 0 

20 has no potting. However in this embodiment, a wall 
2222 would be placed between the lens 22 02 and LED 1 s 
2102 to avoid direct reflection on the filter by the 
LED' s. 

• The camera lens 2202 and camera circuit 2204 
25 are manufactured by Sony Corp. as a 1/3 CCD, 12 VDC 
Board Camera, Model No. UL-BC460/T8. 
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Referring now to Fig. 23, a schematic circuit 
board layout 2300 is shown for the LED array 2100 
(elements D1-D8) . In this arrangement, the lighting 
for diodes D1-D8 is continuous but has a quick "on-off M 
5 to cover a video view. This on-off cycle is 

approximately l/30th of a second. Moreover, the flash 
component of the video view period is l/7000 tn of a 
second. Since the period of lighting is so brief, the 
flash and the lighting exposure render sensitivity to 

10 movement of the subject pratically irrelevant. Flash 
nonetheless is essential since in security 
applications, movement of the subject occurs 
frequently. However, the flash can be changed to a 
continuous lighting mode, if desired. 

15 Each of the IR LED's is a focused beam diode, 

which improves efficiency and also reduces power 
consumption. As noted previously, pin connections 2301 
are adapted to connect directly into a personal 
computer I/O port. 

20 Fig. 24 is an illustration of the circuitry 

supporting the camera electronics 2400. A constant 
power source of about 100 million amps is provided. A 
12 volt power supply is used along with a 5V control 
power supply. 

25 As a consequence of the arrangement of FIGS. 

21-24 a micro, camera arrangement for image capture is 
created whereby the lighting is located below the 
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camera. Moreover, the position at which the lighting 
is below such camera is critical, since a subject 
farther than 3 feet away from the lens will not be 
captured. Placement of the camera is also sensitive 
5 since direct sunlight, incandescent or halogen light 
will wash out features. Thus any direct light to the 
camera is problematic. 

Figures 25a-c are different views (front, side, 
and perspective) of a housing designed to contain the 

10 camera-LED unit. A recess 2502 is shown in the unit, 
through which the entire housing 2200 can be inserted. 
The modular plug 2220 (Fig. 22) would also be connected 
through cable 2504 (Fig. 25c) to the PC I/O port (not 
shown). In the side view of Fig. 25(b), the housing 

15 2504 includes a stand 2506 which pivots about axle 2508 
in the direction of arrow 2510. As a consequence, the 
camera can be supported in a substantially upright 
position (Fig. 25(c)) when placed with the stand in an 
extended position on a horizontal surface. 

20 Figs. 26a-c show a second embodiment of the 

mini-camera housing 2600. As shown, the housing 
includes a stand 2602, which in a closed position (as 
shown in Figs. 26a, 26b), completely covers the camera 
lens and LED's. When fully opened, however, which is 

25 accomplished by rotating the stand 2602 about axis 

2604," the camera, and LED light unit are fully exposed, - 

and is also supported upright by stand 2602 (Fig. 25c) . 
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Finally, Figs. 27a-c are views of a third 
embodiment of the minicamera housing 2700. In this 
embodiment, a stand 2702 is partially cut away, to 
expose the camera lens only. However, when the stand 
5 is opened (Fig. 27(c)) the LED array and the camera 
. 2202 are both exposed for use. 

Fig. 28 illustrates a second or alternative 
embodiment for the targeting process set forth in Fig. 
6 of this invention. The advantage of the alternative 
10 technique is that it allows targeting without reference 
to fixed areas, by dynamically finding the image 
centers. 

The process, 2800 begins at step 2802 where a 
desired area is isolated and captured by the camera. A 

15 histogram for this captured area is then computed by 
the computer 40. The computer then dynamically 
determines thresholds by calculating desired threshold 
barriers that are preprogrammed into the computer 40. 
For example, high and low rejects can be set to be 

20 above the lowest 5% and below the highest 5%, and high 
and low thresholds between the bottom 45% and below the 
top 45%. As a consequence, when the threshold is 
compared to the histogram at step 2808, a 10% middle 
portion of the histogram can be defined reflecting 

25 particular gray-scale characteristics. 

The below, between, and above threshold values.. 

are then binarized at binarization step 2810 as shown 
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in 28 (c) . The first binarization step is the threshold 
comparison itself, which sets values as follows: 
where: HT = high threshold 
LT = low threshold 
5 any value > HT=1 

any value < HT=0 
any value < LT=1 
any value > LT=0 
the binarized values are then compared, and a majority 
10 fill, as shown in Fig. 28(d) occurs, which represents the 
true binarized area of the targeted object 2812. 

The targeted area is then geometrically tested 
at step 2814 on two candidate points based on preset 
values which define appropriate quadrants. The points 
15 *i,Yi and X 2'Y2 can be isolated based on preset template 
values. For example, if iris targeting is desired, eye 
templates can be set so that |X 2 -X,|>40 and IY.-YjKIO. 
Once correlation of the templates has occurred at step 
2816, the image is divided, and the« lower portions of 
20 the image are dropped-out, leaving only the 
side-by-side quadrants . 

In the event that targeting cannot locate the 
desired targets with the appropriate threshold 
characteristics, the system can keep expanding the 
25 image area in order to locate the desired points. 
Specifically an iteration loop can take three (3) 
images, binarize those values, average those binarized 
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value and store the averaged value in the portable 
memory. As a result of this iteration process shown at 
steps 2822 and 2824, a high percentage of accuracy is 
achieved dynamically. 
5 The above description and drawings are only 

illustrative of preferred embodiments which achieve the 
objects, features and advantages of the present 
invention, and it is not intended that the present 
invention be limited thereto. Any modifications of the 
10 present invention which comes within the spirit and 

scope of the following claims is considered part of the 
present invention . 
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We claim : 

1. A system and method for obtaining, 
digitizing, and recording selected data sets of 
separate and independent component element image 

5 signals, sensed by photographic or electronic means, 

from a person or object, for subsequent comparison with 
similarly obtained component elements image signals of 
a person or object whereby the degree of similarity 
between, the original and the subsequent data sets may 
10 be determined. 

2. A method, as in claim 1, whereby data sets 
of sensed images may be recorded on portable media, 
such as magnetic stripe cards, magnetic discs, printed 
bar codes on paper or similar materials, semi-conductor 

15 devices such as smart cards, or in data bases. 

3. A method, as in claim 1, of selecting 
separate and independent component elements of persons 
or objects whereby the selected components provide 
image signals for recording quantitative structure or 

20 array of numbers that permit statistical identification 
of the individual person or object from among data sets 
previously obtained from the same person or similar 
ob j ects . 

4. A method, as in claim 3, whereby the image 
25 signal data sets of persons or objects are measured so 

as to permit the determination of a quantitative degree 
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to which the objects or persons are similar to those 
originally measured* 

5. A method as, in claim 3, whereby it may be 
determined whether component image signals obtained 

5 from a person or object, at any time subsequent to the 
recording of a set of such image signals from the 
person or object, differ, or do not differ, from the 
recorded component images by any, or by more than or 
less than, preselected quantitative values, permitting 
10 the system to indicate that the signals may be 

quantitatively, or qualitatively, verified, or not 
verified, as being from the recorded signals of the 
person or object. 

6. A system, as in claim 1, whereby 

15 quantitative measures may be selected for inclusion in 
the digital data system for allowing the determination 
of chosen levels of difference between subsequently 
obtained component image signals and verification of a 
such signals as being, or not being, from the recorded 

20 signals of a specific person or object. 
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The present system identifies an object, 
such as a face, from unique signals and values 
emitted by the attributes of components on 
the object such as eyes, eyebrows, hairline, 
mouth, cheeks, ears, and chin (22). The object 
image is analyzed to determine the different 
components (325). These components are 
then transformed (340) through one of several 
transformations such as geometric, cosine, or 
Kohonen's, to yield a unique description for 
each component. The data obtained is then 
transferred to either a portable memory, ID 
card or data base. A decision process of 
comparing the two results, from the object 
image, and from the data encoded in the ID 
card or data base is subsequently performed. 
In one example, this technology identifies 
human faces to verify the authenticity of an 
identity and then to grant or deny authority to 
carry out actions. 



ENROLLMENT 



0NE-0N-0NE 



! enrollment 




ocip^WQ — aZ055Q3&iJQ>_xtte No. 15/1997. Section III »• a ;„ v>r-r rwtr* m« 1Q/1007 in 



FOR THE PURPOSES OF INFORMATION ONLY 

Codes used to identify States party to the PCT on the front pages of pamphlets publishing international 
applications under the PCT. 



AM 


Armenia 


GB 


United Kingdom 


MW 


Malawi 


AT 


Austria 


GE 


Georgia 


MX 


Mexico 


AU 


Australia 


GN 


Guinea 


NE 


NigeT 


BB 


Barbados 


GR 


Greece 


NL 


Netherlands 


BE 


Belgium 


HU 


Hungary 


NO 


Norway 


BF 


Burkina Faso 


IE 


Ireland 


NZ 


New Zealand 


BG 


Bulgaria 


IT 


Italy 


PL 


Poland 


BJ 


Benin 


JP 


Japan 


PT 


Portugal 


BR 


Brazil 


KE 


Kenya 


RO 


Romania 


BY 


Belarus 


KG 


Kyrgystan 


RU 


Russian Federation 


CA 


Canada 


KP 


Democratic People's Republic 


SD 


Sudan 


CF 


Central African Republic 




of Korea 


SE 


Sweden 


CC 


Congo 


KR 


Republic of Korea 


SG 


Singapore 


CH 


Switzerland 


KZ 


Kazakhstan 


SI 


Slovenia 


CI 


Cote d'l voire 


U 


Liechtenstein 


SK 


Slovakia 


CM 


Cameroon 


LK 


Sri Lanka 


SN 


Senegal 


CN 


China 


LR 


Liberia 


sz 


Swaziland 


CS 


- Czechoslovakia - — 


LT - 


- Lithuania 


TD 


Chad 


CZ 


Czech Republic 


LU 


Luxembourg 


TG 


Togo 


DE 


Germany 


LV 


Latvia 


TJ 


Tajikistan 


DK 


Denmark 


MC 


Monaco 


TT 


Trinidad and Tobago 


EE 


Estonia 


MD 


Republic of Moldova 


UA 


Ukraine 


ES 


Spain 


MG 


Madagascar 


UG 


Uganda 


Fl 


Finland. 


ML 


Mali 


US 


United States of America 


FR 


France 


MN 


Mongolia 


HZ 


Uzbekistan 


GA 


Gabon 


MR 


Mauritania 


VN 


Viet Nam 



XXID: <WO 9705566A1_IB> 



